250 avsnitt • Längd: 60 min • Veckovis: Torsdag
Audio narrations of LessWrong posts by zvi
The podcast LessWrong posts by zvi is created by zvi. The podcast and the artwork on this page are embedded on this page using the public podcast feed (RSS).
It doesn’t look good.
What used to be the AI Safety Summits were perhaps the most promising thing happening towards international coordination for AI Safety.
This one was centrally coordination against AI Safety.
In November 2023, the UK Bletchley Summit on AI Safety set out to let nations coordinate in the hopes that AI might not kill everyone. China was there, too, and included.
The practical focus was on Responsible Scaling Policies (RSPs), where commitments were secured from the major labs, and laying the foundations for new institutions.
The summit ended with The Bletchley Declaration (full text included at link), signed by all key parties. It was the usual diplomatic drek, as is typically the case for such things, but it centrally said there are risks, and so we will develop policies to deal with those risks.
And it ended with a commitment [...]
---
Outline:
(02:03) An Actively Terrible Summit Statement
(05:45) The Suicidal Accelerationist Speech by JD Vance
(14:37) What Did France Care About?
(17:12) Something To Remember You By: Get Your Safety Frameworks
(24:05) What Do We Think About Voluntary Commitments?
(27:29) This Is the End
(36:18) The Odds Are Against Us and the Situation is Grim
(39:52) Don't Panic But Also Face Reality
The original text contained 4 images which were described by AI.
---
First published:
February 12th, 2025
Source:
https://www.lesswrong.com/posts/qYPHryHTNiJ2y6Fhi/the-paris-ai-anti-safety-summit
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Not too long ago, OpenAI presented a paper on their new strategy of Deliberative Alignment.
The way this works is that they tell the model what its policies are and then have the model think about whether it should comply with a request.
This is an important transition, so this post will go over my perspective on the new strategy.
Note the similarities, and also differences, with Anthropic's Constitutional AI.
How Deliberative Alignment Works
We introduce deliberative alignment, a training paradigm that directly teaches reasoning LLMs the text of human-written and interpretable safety specifications, and trains them to reason explicitly about these specifications before answering.
We used deliberative alignment to align OpenAI's o-series models, enabling them to use chain-of-thought (CoT) reasoning to reflect on user prompts, identify relevant text from OpenAI's internal policies, and draft safer responses.
Our approach achieves highly precise [...]
---
Outline:
(00:29) How Deliberative Alignment Works
(03:27) Why This Worries Me
(07:49) For Mundane Safety It Works Well
The original text contained 3 images which were described by AI.
---
First published:
February 11th, 2025
Source:
https://www.lesswrong.com/posts/CJ4yywLBkdRALc4sT/on-deliberative-alignment
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Scott Alexander famously warned us to Beware Trivial Inconveniences.
When you make a thing easy to do, people often do vastly more of it.
When you put up barriers, even highly solvable ones, people often do vastly less.
Let us take this seriously, and carefully choose what inconveniences to put where.
Let us also take seriously that when AI or other things reduce frictions, or change the relative severity of frictions, various things might break or require adjustment.
This applies to all system design, and especially to legal and regulatory questions.
Table of Contents
---
Outline:
(00:40) Levels of Friction (and Legality)
(02:24) Important Friction Principles
(05:01) Principle #1: By Default Friction is Bad
(05:23) Principle #3: Friction Can Be Load Bearing
(07:09) Insufficient Friction On Antisocial Behaviors Eventually Snowballs
(08:33) Principle #4: The Best Frictions Are Non-Destructive
(09:01) Principle #8: The Abundance Agenda and Deregulation as Category 1-ification
(10:55) Principle #10: Ensure Antisocial Activities Have Higher Friction
(11:51) Sports Gambling as Motivating Example of Necessary 2-ness
(13:24) On Principle #13: Law Abiding Citizen
(14:39) Mundane AI as 2-breaker and Friction Reducer
(20:13) What To Do About All This
The original text contained 1 image which was described by AI.
---
First published:
February 10th, 2025
Source:
https://www.lesswrong.com/posts/xcMngBervaSCgL9cu/levels-of-friction
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
This week we got a revision of DeepMind's safety framework, and the first version of Meta's framework. This post covers both of them.
Table of Contents
Here are links for previous coverage of: DeepMind's Framework 1.0, OpenAI's Framework and Anthropic's Framework.
Meta's RSP (Frontier AI Framework)
Since there is a law saying no two companies can call these documents by the same name, Meta is here to offer us its Frontier AI Framework, explaining how Meta is going to keep us safe while deploying frontier AI systems.
I will say up front, if it sounds like I’m not giving Meta the benefit of the doubt here, it's because I am absolutely not giving Meta the benefit of [...]
---
Outline:
(00:14) Meta's RSP (Frontier AI Framework)
(16:10) DeepMind Updates its Frontier Safety Framework
(31:05) What About Risk Governance
(33:42) Where Do We Go From Here?
The original text contained 12 images which were described by AI.
---
First published:
February 7th, 2025
Source:
https://www.lesswrong.com/posts/etqbEF4yWoGBEaPro/on-the-meta-and-deepmind-safety-frameworks
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
I remember that week I used r1 a lot, and everyone was obsessed with DeepSeek.
They earned it. DeepSeek cooked, r1 is an excellent model. Seeing the Chain of Thought was revolutionary. We all learned a lot.
It's still #1 in the app store, there are still hysterical misinformed NYT op-eds and and calls for insane reactions in all directions and plenty of jingoism to go around, largely based on that highly misleading $6 millon cost number for DeepSeek's v3, and a misunderstanding of how AI capability curves move over time.
But like the tariff threats that's now so yesterday now, for those of us that live in the unevenly distributed future.
All my reasoning model needs go through o3-mini-high, and Google's fully unleashed Flash Thinking for free. Everyone is exploring OpenAI's Deep Research, even in its early form, and I finally have an entity [...]
---
Outline:
(01:15) Language Models Offer Mundane Utility
(07:23) o1-Pro Offers Mundane Utility
(10:35) We're in Deep Research
(17:08) Language Models Don't Offer Mundane Utility
(17:49) Model Decision Tree
(20:43) Huh, Upgrades
(21:57) Bot Versus Bot
(24:04) The OpenAI Unintended Guidelines
(26:40) Peter Wildeford on DeepSeek
(29:18) Our Price Cheap
(35:25) Otherwise Seeking Deeply
(44:13) Smooth Operator
(46:46) Have You Tried Not Building An Agent?
(51:58) Deepfaketown and Botpocalypse Soon
(54:56) They Took Our Jobs
(01:08:29) The Art of the Jailbreak
(01:08:56) Get Involved
(01:13:05) Introducing
(01:13:45) In Other AI News
(01:16:37) Theory of the Firm
(01:21:32) Quiet Speculations
(01:24:36) The Quest for Sane Regulations
(01:33:33) The Week in Audio
(01:34:41) Rhetorical Innovation
(01:38:22) Aligning a Smarter Than Human Intelligence is Difficult
(01:40:33) The Alignment Faking Analysis Continues
(01:44:24) Masayoshi Son Follows Own Advice
(01:48:22) People Are Worried About AI Killing Everyone
(01:50:32) You Are Not Ready
(02:00:45) Other People Are Not As Worried About AI Killing Everyone
(02:02:53) The Lighter Side
The original text contained 22 images which were described by AI.
---
First published:
February 6th, 2025
Source:
https://www.lesswrong.com/posts/rAaGbh7w52soCckNC/ai-102-made-in-america
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
The baseline scenario as AI becomes AGI becomes ASI (artificial superintelligence), if nothing more dramatic goes wrong first and even we successfully ‘solve alignment’ of AI to a given user and developer, is the ‘gradual’ disempowerment of humanity by AIs, as we voluntarily grant them more and more power in a vicious cycle, after which AIs control the future and an ever-increasing share of its real resources. It is unlikely that humans survive it for long.
This gradual disempowerment is far from the only way things could go horribly wrong. There are various other ways things could go horribly wrong earlier, faster and more dramatically, especially if we indeed fail at alignment of ASI on the first try.
Gradual disempowerment it still is a major part of the problem, including in worlds that would otherwise have survived those other threats. And I don’t know of any good [...]
---
Outline:
(01:15) We Finally Have a Good Paper
(02:30) The Phase 2 Problem
(05:02) Coordination is Hard
(07:59) Even Successful Technical Solutions Do Not Solve This
(08:58) The Six Core Claims
(14:35) Proposed Mitigations Are Insufficient
(19:58) The Social Contract Will Change
(21:07) Point of No Return
(22:51) A Shorter Summary
(24:13) Tyler Cowen Seems To Misunderstand Two Key Points
(25:53) Do You Feel in Charge?
(28:04) We Will Not By Default Meaningfully 'Own' the AIs For Long
(29:53) Collusion Has Nothing to Do With This
(32:38) If Humans Do Not Successfully Collude They Lose All Control
(34:45) The Odds Are Against Us and the Situation is Grim
---
First published:
February 5th, 2025
Source:
https://www.lesswrong.com/posts/jEZpfsdaX2dBD9Y6g/the-risk-of-gradual-disempowerment-from-ai
Narrated by TYPE III AUDIO.
Table of Contents
The Pitch
OpenAI: Today we’re launching deep research in ChatGPT, a new agentic capability that conducts multi-step research on the internet for complex tasks. It accomplishes in tens of minutes what would take a human many hours. Sam Altman: Today we launch Deep Research, our next agent. This is like a superpower; experts on [...]---
Outline:
(00:20) The Pitch
(03:12) It's Coming
(05:01) Is It Safe?
(09:49) How Does Deep Research Work?
(10:47) Killer Shopping App
(12:17) Rave Reviews
(18:33) Research Reports
(31:21) Perfecting the Prompt
(32:26) Not So Fast!
(35:46) What's Next?
(36:59) Paying the Five
(37:59) The Lighter Side
The original text contained 4 images which were described by AI.
---
First published:
February 4th, 2025
Source:
https://www.lesswrong.com/posts/QqSxKRKJupjuDkymQ/we-re-in-deep-research
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
New model, new hype cycle, who dis?
On a Friday afternoon, OpenAI was proud to announce the new model o3-mini and also o3-mini-high which is somewhat less mini, or for some other reasoning tasks you might still want o1 if you want a broader knowledge base, or if you’re a pro user o1-pro, while we want for o3-not-mini and o3-pro, except o3 can use web search and o1 can’t so it has the better knowledge in that sense, then on a Sunday night they launched Deep Research which is different from Google's Deep Research but you only have a few of those queries so make them count, or maybe you want to use operator?
Get it? Got it? Good.
Yes, Pliny jailbroke o3-mini on the spot, as he always does.
This most mostly skips over OpenAI's Deep Research (o3-DR? OAI-DR?). I need more time for [...]
---
Outline:
(01:16) Feature Presentation
(04:37) QandA
(09:14) The Wrong Side of History
(13:29) The System Card
(22:08) The Official Benchmarks
(24:55) The Unofficial Benchmarks
(27:43) Others Report In
(29:47) Some People Need Practical Advice
The original text contained 10 images which were described by AI.
---
First published:
February 3rd, 2025
Source:
https://www.lesswrong.com/posts/srdxEAcdmetdAiGcz/o3-mini-early-days
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
As reactions continue, the word in Washington, and out of OpenAI, is distillation. They’re accusing DeepSeek of distilling o1, of ripping off OpenAI. They claim DeepSeek *gasp* violated the OpenAI Terms of Service! The horror.
And they are very cross about this horrible violation, and if proven they plan to ‘aggressively treat it as theft,’ while the administration warns that we must put a stop to this.
Aside from the fact that this is obviously very funny, and that there is nothing they could do about it in any case, is it true?
Meanwhile Anthropic's Dario Amodei offers a reaction essay, which also includes a lot of good technical discussion of why v3 and r1 aren’t actually all that unexpected along the cost and capability curves over time, calling for America to race towards AGI to gain decisive strategic advantage over China via recursive self-improvement, although [...]
---
Outline:
(01:01) Seeking Deeply
(01:41) The Market Is In DeepSeek
(06:42) Machines Not of Loving Grace
(17:49) The Kinda Six Million Dollar Model
(18:59) v3 Implies r1
(20:32) Two Can Play That Game
(21:21) Janus Explores r1's Chain of Thought Shenanigans
(24:42) In Other DeepSeek and China News
(27:29) The Quest for Sane Regulations
(29:53) Copyright Confrontation
(37:24) Vibe Gap
(41:09) Deeply Seeking Safety
(42:15) Deeply Seeking Robotics
(45:14) Thank You For Your Candor
(48:21) Thank You For Your Understanding
(51:17) The Lighter Side
The original text contained 1 footnote which was omitted from this narration.
The original text contained 7 images which were described by AI.
---
First published:
January 31st, 2025
Source:
https://www.lesswrong.com/posts/Cc2TagjY2pGAhn7MZ/deepseek-don-t-panic
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
The avalanche of DeepSeek news continues. We are not yet spending more than a few hours at a time in the singularity, where news happens faster than it can be processed. But it's close, and I’ve had to not follow a bunch of other non-AI things that are also happening, at least not well enough to offer any insights.
So this week we’re going to consider China, DeepSeek and r1 fully split off from everything else, and we’ll cover everything related to DeepSeek, including the policy responses to the situation, tomorrow instead.
This is everything else in AI from the past week. Some of it almost feels like it is from another time, so long ago.
I’m afraid you’re going to need to get used to that feeling.
Also, I went on Odd Lots to discuss DeepSeek, where I was and truly hope to again [...]
---
Outline:
(00:55) Language Models Offer Mundane Utility
(02:47) Language Models Don't Offer Mundane Utility
(05:43) Language Models Don't Offer You In Particular Mundane Utility
(10:49) (Don't) Feel the AGI
(12:36) Huh, Upgrades
(16:08) They Took Our Jobs
(21:30) Get Involved
(22:04) Introducing
(23:38) In Other AI News
(27:10) Hype
(29:56) We Had a Deal
(31:43) Quiet Speculations
(37:14) The Quest for Sane Regulations
(39:40) The Week in Audio
(39:51) Don't Tread on Me
(45:42) Rhetorical Innovation
(55:22) Scott Sumner on Objectivity in Taste, Ethics and AGI
(01:04:41) The Mask Comes Off (1)
(01:06:58) The Mask Comes Off (2)
(01:09:12) International AI Safety Report
(01:10:37) One Step at a Time
(01:14:12) Aligning a Smarter Than Human Intelligence is Difficult
(01:18:54) Two Attractor States
(01:26:51) You Play to Win the Game
(01:28:10) Six Thoughts on AI Safety
(01:35:53) AI Situational Awareness
(01:40:15) People Are Worried About AI Killing Everyone
(01:43:40) Other People Are Not As Worried About AI Killing Everyone
(01:44:24) The Lighter Side
The original text contained 17 images which were described by AI.
---
First published:
January 30th, 2025
Source:
https://www.lesswrong.com/posts/pZ6htFtoptGrSajWG/ai-101-the-shallow-end
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
It's been another *checks notes* two days, so it's time for all the latest DeepSeek news.
You can also see my previous coverage of the r1 model and, from Monday various reactions including the Panic at the App Store.
Table of Contents
First, Reiterating About Calming Down About the [...]
---
Outline:
(00:20) First, Reiterating About Calming Down About the $5.5 Million Number
(01:58) OpenAI Offers Its Congratulations
(05:54) Scaling Laws Still Apply
(12:07) Other r1 and DeepSeek News Roundup
(16:06) People Love Free
(18:45) Investigating How r1 Works
(23:40) Nvidia Chips are Highly Useful
(24:52) Welcome to the Market
(30:02) Ben Thompson Weighs In
(33:01) Import Restrictions on Chips WTAF
(35:37) Are You Short the Market
(39:52) DeepSeeking Safety
(43:11) Mo Models Mo Problems
(50:23) What If You Wanted to Restrict Already Open Models
(53:33) So What Are We Going to Do About All This?
The original text contained 5 images which were described by AI.
---
First published:
January 29th, 2025
Source:
https://www.lesswrong.com/posts/jzjph4yYtgAsLeWmg/deepseek-lemon-it-s-wednesday
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
No one is talking about OpenAI's Operator. We’re, shall we say, a bit distracted.
It's still a rather meaningful thing that happened last week. I too have been too busy to put it through its paces, but this is the worst it will ever be, and the least available and most expensive it will ever be. The year of the agent is indeed likely coming.
So, what do we have here?
Hello, Operator
OpenAI has introduced the beta for its new agent, called Operator, which is now live for Pro users and will in the future be available to Plus users, ‘with more agents to launch in the coming weeks and months.’
Here is a 22 minute video demo. Here is the system card.
You start off by optionally specifying a particular app (in the first demo, OpenTable) and then give it a [...]
---
Outline:
(00:28) Hello, Operator
(02:44) Risky Operation
(04:34) Basic Training
(06:29) Please Stay on the Line
(12:08) For a Brief Survey
(16:34) The Number You Are Calling Is Not Available (In the EU)
(17:20) How to Get Ahead in Advertising
(19:05) Begin Operation
(20:11) The Lighter Side
The original text contained 4 images which were described by AI.
---
First published:
January 28th, 2025
Source:
https://www.lesswrong.com/posts/jTtbnSyS9knzZehCm/operator
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
DeepSeek released v3. Market didn’t react.
DeepSeek released r1. Market didn’t react.
DeepSeek released a f***ing app of its website. Market said I have an idea, let's panic.
Nvidia was down 11%, Nasdaq is down 2.5%, S&P is down 1.7%, on the news.
Shakeel: The fact this is happening today, and didn’t happen when r1 actually released last Wednesday, is a neat demonstration of how the market is in fact not efficient at all.
That is exactly the market's level of situational awareness. No more, no less.
I traded accordingly. But of course nothing here is ever investment advice.
Given all that has happened, it seems worthwhile to go over all the DeepSeek news that has happened since Thursday. Yes, since Thursday.
For previous events, see my top level post here, and additional notes on Thursday.
To avoid confusion: r1 [...]
---
Outline:
(01:27) Current Mood
(03:04) DeepSeek Tops the Charts
(07:42) Why Is DeepSeek Topping the Charts?
(09:47) What Is the DeepSeek Business Model?
(13:48) The Lines on Graphs Case for Panic
(16:31) Everyone Calm Down About That $5.5 Million Number
(25:42) Is The Whale Lying?
(29:33) Capex Spending on Compute Will Continue to Go Up
(32:53) Jevon's Paradox Strikes Again
(36:24) Okay, Maybe Meta Should Panic
(39:02) Are You Short the Market
(43:52) o1 Versus r1
(47:23) Additional Notes on v3 and r1
(50:12) Janus-Pro-7B Sure Why Not
(50:44) Man in the Arena
(52:42) Training r1, and Training With r1
(56:34) Also Perhaps We Should Worry About AI Killing Everyone
(59:21) And We Should Worry About Crazy Reactions To All This, Too
(01:02:14) The Lighter Side
The original text contained 10 images which were described by AI.
---
First published:
January 28th, 2025
Source:
https://www.lesswrong.com/posts/hRxGrJJq6ifL4jRGa/deepseek-panic-at-the-app-store
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
There was a comedy routine a few years ago. I believe it was by Hannah Gadsby. She brought up a painting, and looked at some details. The details weren’t important in and of themselves. If an AI had randomly put them there, we wouldn’t care.
Except an AI didn’t put them there. And they weren’t there at random.
A human put them there. On purpose. Or, as she put it:
THAT was a DECISION.
This is the correct way to view decisions around a $500 billion AI infrastructure project, announced right after Trump takes office, having it be primarily funded by SoftBank, with all the compute intended to be used by OpenAI, and calling it Stargate.
Table of Contents
---
Outline:
(00:49) The Announcement
(05:21) Is That a Lot?
(09:37) What Happened to the Microsoft Partnership?
(11:03) Where's Our 20%?
(12:10) Show Me the Money
(17:23) It Never Hurts to Suck Up to the Boss
(24:44) What's in a Name
(29:01) Just Think of the Potential
(34:00) I Believe Toast is an Adequate Description
(36:05) The Lighter Side
The original text contained 5 images which were described by AI.
---
First published:
January 24th, 2025
Source:
https://www.lesswrong.com/posts/fwt7ojAb6zgEaLJMB/stargate-ai-1
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Break time is over, it would seem, now that the new administration is in town.
This week we got r1, DeepSeek's new reasoning model, which is now my go-to first choice for a large percentage of queries. The claim that this was the most important thing to happen on January 20, 2025 was at least non-crazy. If you read about one thing this week read about that.
We also got the announcement of Stargate, a claimed $500 billion private investment in American AI infrastructure. I will be covering that on its own soon.
Due to time limits I have also pushed coverage of a few things into next week, including this alignment paper, and I still owe my take on Deliberative Alignment.
The Trump administration came out swinging on many fronts with a wide variety of executive orders. For AI, that includes repeal of the [...]
---
Outline:
(01:24) Language Models Offer Mundane Utility
(10:54) Language Models Don't Offer Mundane Utility
(17:20) Huh, Upgrades
(20:03) Additional Notes on r1
(22:41) Fun With Media Generation
(23:18) We Tested Older LLMs and Are Framing It As a Failure
(26:56) Deepfaketown and Botpocalypse Soon
(32:10) They Took Our Jobs
(47:15) Get Involved
(47:54) Introducing
(51:38) We Had a Deal
(01:07:17) In Other AI News
(01:18:39) Whistling in the Dark
(01:22:03) Quiet Speculations
(01:28:09) Suchir's Last Post
(01:29:43) Modeling Lower Bound Economic Growth From AI
(01:34:42) The Quest for Sane Regulations
(01:39:53) The Week in Audio
(01:42:51) Rhetorical Innovation
(01:49:37) Cry Havoc
(01:53:14) Aligning a Smarter Than Human Intelligence is Difficult
(01:59:34) People Strongly Dislike AI
(02:02:23) People Are Worried About AI Killing Everyone
(02:05:17) Other People Are Not As Worried About AI Killing Everyone
(02:09:29) The Lighter Side
The original text contained 16 images which were described by AI.
---
First published:
January 23rd, 2025
Source:
https://www.lesswrong.com/posts/PjDjeGPYPoi9qfPr2/ai-100-meet-the-new-boss
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
r1 from DeepSeek is here, the first serious challenge to OpenAI's o1.
r1 is an open model, and it comes in dramatically cheaper than o1.
People are very excited. Normally cost is not a big deal, but o1 and its inference-time compute strategy is the exception. Here, cheaper really can mean better, even if the answers aren’t quite as good.
You can get DeepSeek-r1 on HuggingFace here, and they link to the paper.
The question is how to think about r1 as it compares to o1, and also to o1 Pro and to the future o3-mini that we’ll get in a few weeks, and then to o3 which we’ll likely get in a month or two.
Taking into account everything I’ve seen, r1 is still a notch below o1 in terms of quality of output, and further behind o1 Pro and the future o3-mini [...]
---
Outline:
(01:43) Part 1: RTFP: Read the Paper
(03:38) How Did They Do It
(06:19) The Aha Moment
(08:27) Benchmarks
(09:46) Reports of Failure
(11:11) Part 2: Capabilities Analysis
(11:16) Our Price Cheap
(15:44) Other People's Benchmarks
(18:20) r1 Makes Traditional Silly Mistakes
(23:11) The Overall Vibes
(25:36) If I Could Read Your Mind
(28:06) Creative Writing
(32:21) Bring On the Spice
(34:33) We Cracked Up All the Censors
(39:44) Switching Costs Are Low In Theory
(42:15) The Self-Improvement Loop
(44:18) Room for Improvement
(48:27) Part 3: Where Does This Leave Us on Existential Risk?
(48:58) The Suicide Caucus
(51:21) v3 Implies r1
(53:09) Open Weights Are Unsafe And Nothing Can Fix This
(58:59) So What the Hell Should We Do About All This?
(01:05:53) Part 4: The Lighter Side
The original text contained 20 images which were described by AI.
---
First published:
January 22nd, 2025
Source:
https://www.lesswrong.com/posts/buTWsjfwQGMvocEyw/on-deepseek-s-r1
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
As always, some people need practical advice, and we can’t agree on how any of this works and we are all different and our motivations are different, so figuring out the best things to do is difficult. Here are various hopefully useful notes.
Table of Contents
Effectiveness of GLP-1 Drugs
GLP-1 drugs are so effective that the American obesity rate is falling.
John Burn-Murdoch: While we can’t be certain that the [...]
---
Outline:
(00:22) Effectiveness of GLP-1 Drugs
(01:09) What Passes for Skepticism on GLP-1s
(03:21) The Joy of Willpower
(10:07) Talking Supply
(10:44) Talking Price
(13:36) GLP-1 Inhibitors Help Solve All Your Problems
(14:12) Dieting the Hard Way
(18:41) Nutrients
(19:46) Are Vegetables a Scam?
(22:46) Government Food Labels Are Often Obvious Nonsense
(23:33) Sleep
(28:34) Find a Way to Enjoy Exercise
(32:28) A Note on Alcohol
(33:04) Focus Only On What Matters
The original text contained 2 images which were described by AI.
---
First published:
January 21st, 2025
Source:
https://www.lesswrong.com/posts/YLi47gRquTJqLsgoe/sleep-diet-exercise-and-glp-1-drugs
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
There's going to be some changes made.
Table of Contents
Out With the Fact Checkers
Mark Zuckerberg has decided that with Donald Trump soon to be in office, he is allowed to care about free speech again. And he has decided it is time to admit that what was called ‘fact checking’ meant he had for years been running a giant hugely biased, trigger-happy and error prone left-wing censorship and moderation machine that had standards massively out of touch with ordinary people and engaged in automated taking down of often innocent accounts.
He also admits that the majority of censorship in the past has flat out [...]
---
Outline:
(00:08) Out With the Fact Checkers
(01:21) What Happened
(04:05) Timing is Everything
(05:29) Balancing Different Errors
(06:25) Truth and Reconciliation
(08:06) Fact Check Fact Check
(11:33) Mistakes Will Be Made
(16:11) Where We Go From Here
The original text contained 2 images which were described by AI.
---
First published:
January 17th, 2025
Source:
https://www.lesswrong.com/posts/Mdeszo3C44qEAXB8y/meta-pivots-on-content-moderation
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
The fun, as it were, is presumably about to begin.
And the break was fun while it lasted.
Biden went out with an AI bang. His farewell address warns of a ‘Tech-Industrial Complex’ and calls AI the most important technology of all time. And there was not one but two AI-related everything bagel concrete actions proposed – I say proposed because Trump could undo or modify either or both of them.
One attempts to build three or more ‘frontier AI model data centers’ on federal land, with timelines and plans I can only summarize with ‘good luck with that.’ The other move was new diffusion regulations on who can have what AI chips, an attempt to actually stop China from accessing the compute it needs. We shall see what happens.
Table of Contents
---
Outline:
(00:53) Language Models Offer Mundane Utility
(06:45) Language Models Don't Offer Mundane Utility
(10:40) What AI Skepticism Often Looks Like
(13:59) A Very Expensive Chatbot
(16:07) Deepfaketown and Botpocalypse Soon
(21:51) Fun With Image Generation
(22:15) They Took Our Jobs
(27:53) The Blame Game
(31:25) Copyright Confrontation
(31:44) The Six Million Dollar Model
(34:51) Get Involved
(35:15) Introducing
(38:36) In Other AI News
(41:32) Quiet Speculations
(53:27) Man With a Plan
(58:40) Our Price Cheap
(01:03:09) The Quest for Sane Regulations
(01:05:54) Super Duper Export Controls
(01:14:17) Everything Bagel Data Centers
(01:20:46) d/acc Round 2
(01:30:42) The Week in Audio
(01:33:57) Rhetorical Innovation
(01:39:32) Aligning a Smarter Than Human Intelligence is Difficult
(01:47:47) Other People Are Not As Worried About AI Killing Everyone
(01:51:03) The Lighter Side
The original text contained 18 images which were described by AI.
---
First published:
January 16th, 2025
Source:
https://www.lesswrong.com/posts/dnqpcq9S7voPwpvRA/ai-99-farewell-to-biden
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Table of Contents
Man With a Plan
The primary Man With a Plan this week for government-guided AI prosperity was UK Prime Minister Keir Starmer, with a plan coming primarily from Matt Clifford. I’ll be covering that soon.
Today I will be covering the other Man With a Plan, Sam Altman, as OpenAI offers its Economic Blueprint.
Cyrps1s (CISO OpenAI): AI is the ultimate race. The winner decides whether the future looks free and democratic, or repressed and authoritarian.
OpenAI, and the Western World, must win – and we have a blueprint to do so.
Do you hear yourselves? The mask on race and jingoism could not be more off, or [...]
---
Outline:
(00:03) Man With a Plan
(01:06) Oh the Pain
(03:47) Actual Proposals
(07:37) For AI Builders
(08:08) Think of the Children
(09:00) Content Identification
(10:53) Infrastructure Week
(14:31) Paying Attention
---
First published:
January 15th, 2025
Source:
https://www.lesswrong.com/posts/uxnKrsgAzKFZDk4bJ/on-the-openai-economic-blueprint
Narrated by TYPE III AUDIO.
Table of Contents
Congestion Pricing Comes to NYC
We’ve now had over a week of congestion pricing in New York City. It took a while to finally get it. The market for whether congestion pricing would happen in 2024 got as high as 87% before Governor Hochul first betrayed us. Fortunately for us, she partially caved. We finally got congestion pricing at the start of 2025. In the end, we got [...]---
Outline:
(00:13) Congestion Pricing Comes to NYC
(02:35) How Much Is Traffic Improving?
(11:24) And That's Terrible?
(14:00) You Mad, Bro
(15:21) All Aboard
(19:02) Time is Money
(20:58) Solving For the Equilibrium
(23:19) Enforcement and License Plates
(25:13) Uber Eats the Traffic
(27:20) We Can Do Even Better Via Congestion Tolls
(29:32) Abundance Agenda Fever Dream
(31:18) The Lighter Side
The original text contained 9 images which were described by AI.
---
First published:
January 14th, 2025
Source:
https://www.lesswrong.com/posts/GN8SrMxw3WEAtfrFS/nyc-congestion-pricing-early-days
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Now that I am tracking all the movies I watch via Letterboxd, it seems worthwhile to go over the results at the end of the year, and look for lessons, patterns and highlights.
Table of Contents
The Rating Scale
Letterboxd [...]
---
Outline:
(00:15) The Rating Scale
(02:37) The Numbers
(03:16) Very Briefly on the Top Picks and Whether You Should See Them
(04:16) Movies Have Decreasing Marginal Returns in Practice
(05:19) Theaters are Awesome
(07:14) I Hate Spoilers With the Fire of a Thousand Suns
(08:32) Scott Sumner Picks Great American Movies Then Dislikes Them
(09:55) I Knew Before the Cards Were Even Turned Over
(11:19) Other Notes to Self to Remember
(12:24) Strong Opinions, Strongly Held: I Didn't Like It
(14:31) Strong Opinions, Strongly Held: I Did Like It
(19:45) Megalopolis
(20:55) The Brutalist
(24:51) The Death of Award Shows
(27:19) On to 2025
The original text contained 2 images which were described by AI.
---
First published:
January 13th, 2025
Source:
https://www.lesswrong.com/posts/6bgAzPqNppojyGL2v/zvi-s-2024-in-movies
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Dwarkesh Patel again interviewed Tyler Cowen, largely about AI, so here we go.
Note that I take it as a given that the entire discussion is taking place in some form of an ‘AI Fizzle’ and ‘economic normal’ world, where AI does not advance too much in capability from its current form, in meaningful senses, and we do not get superintelligence [because of reasons]. It's still massive additional progress by the standards of any other technology, but painfully slow by the ‘AGI is coming soon’ crowd.
That's the only way I can make the discussion make at least some sense, with Tyler Cowen predicting 0.5%/year additional RGDP growth from AI. That level of capabilities progress is a possible world, although the various elements stated here seem like they are sometimes from different possible worlds.
I note that this conversation was recorded prior to o3 and all [...]
---
Outline:
(02:01) AI and Economic Growth
(02:28) Cost Disease
(09:03) The Lump of Intelligence Fallacy
(10:56) The Efficient Market Hypothesis is False
(13:05) Not Sending Your Best People
(20:49) Energy as the Bottleneck
(22:31) The Experts are Wrong But Trust Them Anyway
(25:41) AI as Additional Population
(26:51) Opposition to AI as the Bottleneck
(29:10) China as Existence Proof for Rapid Growth
(29:56) Second Derivatives
(31:05) Talent and Leadership
(33:26) Adapting to the Age of AI
(35:37) Identifying Alpha
(37:39) Old Man Yells at Crowd
(41:59) Some Statements for Everyone to Ponder
(43:13) No Royal Road to Wisdom
(47:29) Concluding Thoughts
The original text contained 1 image which was described by AI.
---
First published:
January 10th, 2025
Source:
https://www.lesswrong.com/posts/esWbhgHd6bcfsTjGL/on-dwarkesh-patel-s-4th-podcast-with-tyler-cowen
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
The world is kind of on fire. The world of AI, in the very short term and for once, is not, as everyone recovers from the avalanche that was December, and reflects.
Altman was the star this week. He has his six word story, and he had his interview at Bloomberg and his blog post Reflections. I covered the later two of those in OpenAI #10, if you read one AI-related thing from me this week that should be it.
Table of Contents
---
Outline:
(00:34) Language Models Offer Mundane Utility
(04:51) Language Models Don't Offer Mundane Utility
(13:25) Power User
(18:04) Locked In User
(19:14) Read the Classics
(23:58) Deepfaketown and Botpocalypse Soon
(24:11) Fun With Image Generation
(25:11) They Took Our Jobs
(25:35) Question Time
(29:43) Get Involved
(30:02) Introducing
(30:28) In Other AI News
(32:34) Quiet Speculations
(37:19) The Quest for Sane Regulations
(38:43) The Least You Could Do
(44:55) Six Word Story
(47:31) The Week in Audio
(47:55) And I Feel Fine
(55:26) Rhetorical Innovation
(58:07) Liar Liar
(59:12) Feel the AGI
(01:01:57) Regular Americans Hate AI
(01:04:18) Aligning a Smarter Than Human Intelligence is Difficult
(01:11:31) The Lighter Side
The original text contained 8 images which were described by AI.
---
First published:
January 9th, 2025
Source:
https://www.lesswrong.com/posts/xkpPLR3S4SASPeTgC/ai-98-world-ends-with-six-word-story
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
This week, Altman offers a post called Reflections, and he has an interview in Bloomberg. There's a bunch of good and interesting answers in the interview about past events that I won’t mention or have to condense a lot here, such as his going over his calendar and all the meetings he constantly has, so consider reading the whole thing.
Table of Contents
The Battle of the Board
Here is what he says about the Battle of the Board in Reflections:
Sam Altman: A little over a year ago, on one particular Friday, the main thing that had gone wrong that day was [...]
---
Outline:
(00:25) The Battle of the Board
(05:12) Altman Lashes Out
(07:48) Inconsistently Candid
(09:35) On Various People Leaving OpenAI
(10:56) The Pitch
(12:07) Great Expectations
(12:56) Accusations of Fake News
(15:02) OpenAI's Vision Would Pose an Existential Risk To Humanity
---
First published:
January 7th, 2025
Source:
https://www.lesswrong.com/posts/XAKYawaW9xkb3YCbF/openai-10-reflections
Narrated by TYPE III AUDIO.
Related: On the 2nd CWT with Jonathan Haidt, The Kids are Not Okay, Full Access to Smartphones is Not Good For Children
It's rough out there. In this post, I’ll cover the latest arguments that smartphones should be banned in schools, including simply because the notifications are too distracting (and if you don’t care much about that, why are the kids in school at all?), problems with kids on social media including many negative interactions, and also the new phenomenon called sextortion.
Table of Contents
How Many Notifications?
Tanagra Beast reruns the experiment of having a class tally their phone notifications. The results were highly compatible with the original experiment.
The tail, it was long.
Ah! So right away we can see [...]
---
Outline:
(00:37) How Many Notifications?
(06:05) Ban Smartphones in Schools
(15:03) Antisocial Media
(19:07) Screen Time
(20:13) Cyberbullying
(21:12) Sextortion
The original text contained 5 images which were described by AI.
---
First published:
January 6th, 2025
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
The Rationalist Project was our last best hope for peace.
An epistemic world 50 million words long, serving as neutral territory.
A place of research and philosophy for 30 million unique visitors
A shining beacon on the internet, all alone in the night.
It was the ending of the Age of Mankind.
The year the Great Race came upon us all.
This is the story of the last of the blogosphere.
The year is 2025. The place is Lighthaven.
As is usually the case, the final week of the year was mostly about people reflecting on the past year or predicting and planning for the new one.
Table of Contents
The most important developments were processing the two new models: OpenAI's o3, and DeepSeek v3.
---
Outline:
(00:45) Language Models Offer Mundane Utility
(07:23) Language Models Don't Offer Mundane Utility
(07:49) Deepfaketown and Botpocalypse Soon
(10:54) Fun With Image Generation
(13:01) They Took Our Jobs
(14:34) Get Involved
(16:23) Get Your Safety Papers
(28:15) Introducing
(28:32) In Other AI News
(30:01) The Mask Comes Off
(40:08) Wanna Bet
(44:44) The Janus Benchmark
(48:07) Quiet Speculations
(59:29) AI Will Have Universal Taste
(01:01:26) Rhetorical Innovation
(01:02:22) Nine Boats and a Helicopter
(01:07:04) Aligning a Smarter Than Human Intelligence is Difficult
(01:13:19) The Lighter Side
The original text contained 3 images which were described by AI.
---
First published:
January 2nd, 2025
Source:
https://www.lesswrong.com/posts/5rDrErovmTyv4duDv/ai-97-4
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
What should we make of DeepSeek v3?
DeepSeek v3 seems to clearly be the best open model, the best model at its price point, and the best model with 37B active parameters, or that cost under $6 million.
According to the benchmarks, it can play with GPT-4o and Claude Sonnet.
Anecdotal reports and alternative benchmarks tells us it's not as good as Claude Sonnet, but it is plausibly on the level of GPT-4o.
So what do we have here? And what are the implications?
Table of Contents
What is DeepSeek v3 Techncially?
I’ve now had a [...]
---
Outline:
(00:39) What is DeepSeek v3 Techncially?
(01:56) Our Price Cheap
(02:33) Run Model Run
(04:57) Talent Search
(05:22) The Amazing Incredible Benchmarks
(07:23) Underperformance on AidanBench
(12:59) Model in the Arena
(13:27) Other Private Benchmarks
(15:05) Anecdata
(23:57) Implications and Policy
The original text contained 9 images which were described by AI.
---
First published:
December 31st, 2024
Source:
https://www.lesswrong.com/posts/NmauyiPBXcGwoArhJ/deekseek-v3-the-six-million-dollar-model
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
OpenAI presented o3 on the Friday before Thanksgiving, at the tail end of the 12 Days of Shipmas.
I was very much expecting the announcement to be something like a price drop. What better way to say ‘Merry Christmas,’ no?
They disagreed. Instead, we got this (here's the announcement, in which Sam Altman says ‘they thought it would be fun’ to go from one frontier model to their next frontier model, yeah, that's what I’m feeling, fun):
Greg Brockman (President of OpenAI): o3, our latest reasoning model, is a breakthrough, with a step function improvement on our most challenging benchmarks. We are starting safety testing and red teaming now.
Nat McAleese (OpenAI): o3 represents substantial progress in general-domain reasoning with reinforcement learning—excited that we were able to announce some results today! Here is a summary of what we shared about o3 in the livestream.
---
Outline:
(03:48) GPQA Has Fallen
(04:21) Codeforces Has Fallen
(05:32) Arc Has Kinda of Fallen But For Now Only Kinda
(09:27) They Trained on the Train Set
(15:26) AIME Has Fallen
(15:58) Frontier of Frontier Math Shifting Rapidly
(19:09) FrontierMath 4: We're Going To Need a Bigger Benchmark
(23:10) What is o3 Under the Hood?
(25:17) Not So Fast!
(28:38) Deep Thought
(30:03) Our Price Cheap
(36:32) Has Software Engineering Fallen?
(37:42) Don't Quit Your Day Job
(40:48) Master of Your Domain
(43:21) Safety Third
(47:56) The Safety Testing Program
(48:58) Safety testing in the reasoning era
(51:01) How to apply
(53:07) What Could Possibly Go Wrong?
(56:36) What Could Possibly Go Right?
(57:06) Send in the Skeptic
(59:25) This is Almost Certainly Not AGI
(01:02:57) Does This Mean the Future is Open Models?
(01:07:17) Not Priced In
(01:08:39) Our Media is Failing Us
(01:14:56) Not Covered Here: Deliberative Alignment
(01:15:08) The Lighter Side
The original text contained 22 images which were described by AI.
---
First published:
December 30th, 2024
Source:
https://www.lesswrong.com/posts/QHtd2ZQqnPAcknDiQ/o3-oh-my
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
The year in models certainly finished off with a bang.
In this penultimate week, we get o3, which purports to give us vastly more efficient performance than o1, and also to allow us to choose to spend vastly more compute if we want a superior answer.
o3 is a big deal, making big gains on coding tests, ARC and some other benchmarks. How big a deal is difficult to say given what we know now. It's about to enter full fledged safety testing.
o3 will get its own post soon, and I’m also pushing back coverage of Deliberative Alignment, OpenAI's new alignment strategy, to incorporate into that.
We also got DeepSeek v3, which claims to have trained a roughly Sonnet-strength model for only $6 million and 37b active parameters per token (671b total via mixture of experts).
DeepSeek v3 gets its own brief section [...]
---
Outline:
(01:25) Language Models Offer Mundane Utility
(04:47) Language Models Don’t Offer Mundane Utility
(06:43) Flash in the Pan
(10:58) The Six Million Dollar Model
(15:50) And I’ll Form the Head
(17:17) Huh, Upgrades
(18:19) o1 Reactions
(23:28) Fun With Image Generation
(25:06) Introducing
(25:52) They Took Our Jobs
(30:20) Get Involved
(30:34) In Other AI News
(34:15) You See an Agent, You Run
(34:58) Another One Leaves the Bus
(35:53) Quiet Speculations
(40:24) Lock It In
(42:34) The Quest for Sane Regulations
(55:35) The Week in Audio
(57:56) A Tale as Old as Time
(01:01:16) Rhetorical Innovation
(01:03:08) Aligning a Smarter Than Human Intelligence is Difficult
(01:04:26) People Are Worried About AI Killing Everyone
(01:06:50) The Lighter Side
The original text contained 17 images which were described by AI.
---
First published:
December 26th, 2024
Source:
https://www.lesswrong.com/posts/k8bkugdhiFmXHPoLH/ai-96-o3-but-not-yet-for-thee
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
This post goes over the important and excellent new paper from Anthropic and Redwood Research, with Ryan Greenblatt as lead author, Alignment Faking in Large Language Models.
This is by far the best demonstration so far of the principle that AIs Will Increasingly Attempt Shenanigans.
This was their announcement thread.
New Anthropic research: Alignment faking in large language models.
In a series of experiments with Redwood Research, we found that Claude often pretends to have different views during training, while actually maintaining its original preferences.
Claude usually refuses harmful queries. We told it we were instead training it to comply with them. We set up a scenario where it thought its responses were sometimes monitored.
When unmonitored, it nearly always complied. But when monitored, it faked alignment 12% of the time.
[thread continues and includes various visual aids.
The AI wanted [...]
---
Outline:
(02:54) The Core Shenanigans in Question
(06:00) Theme and Variations
(07:34) How This Interacts with o3 and OpenAI's Reflective Alignment
(09:17) The Goal Being Plausibly Good Was Incidental
(11:13) Answering Priming Objections
(12:17) What Does Claude Sonnet Think Of This?
(14:07) What Exactly is the Direct Threat Model?
(16:23) RL Training Under Situational Awareness Can Amplify These Behaviors
(20:38) How the Study Authors Updated
(27:08) How Some Others Updated
(42:49) Having the Discussion We Keep Having
(46:49) We Can Now Agree That the Goal is Already There
(47:49) What Would Happen if the Target Was Net Good?
(50:14) But This Was a No Win Situation
(55:52) But Wasn’t It Being a Good Opus? Why Should it be Corrigible?
(01:04:34) Tradeoffs Make The Problem Harder They Don’t Give You a Pass
(01:07:44) But You Told the Model About the Training Procedure
(01:08:35) But the Model is Only Role Playing
(01:09:39) But You Are Saying the Model is a Coherent Person
(01:15:53) But this Headline and Framing Was Misleading
(01:29:22) This Result is Centrally Unsurprising
(01:32:52) Lab Support for Alignment Research Matters
(01:33:50) The Lighter Side
The original text contained 1 footnote which was omitted from this narration.
The original text contained 9 images which were described by AI.
---
First published:
December 24th, 2024
Source:
https://www.lesswrong.com/posts/gHjzdLD6yeLNdsRmw/ais-will-increasingly-fake-alignment
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
I took a trip to San Francisco early in December.
Ever since then, things in the world of AI have been utterly insane.
Google and OpenAI released endless new products, including Google Flash 2.0 and o1.
Redwood Research and Anthropic put out the most important alignment paper of the year, on the heels of Apollo's report on o1.
Then OpenAI announced o3. Like the rest of the media, this blog currently is horrendously lacking in o3 content. Unlike the rest of the media, it is not because I don’t realize that This Changes Everything. It is because I had so much in the queue, and am taking the time to figure out what to think about it.
That queue includes all the other, non-AI things that happened this past month.
So here we are, to kick off Christmas week.
Bad [...]
---
Outline:
(00:57) Bad News
(02:58) What a Lot of Accusations Look Like These Days
(05:47) Good News, Everyone
(08:08) Antisocial Media
(14:53) Government Working
(22:41) Technology Advances
(23:40) I Was Promised Flying Self-Driving Cars
(24:29) For Science!
(25:12) Variously Effective Altruism
(28:37) While I Cannot Condone This
(33:43) Motivation
(34:28) Knowing Better
(36:38) For Your Entertainment
(39:22) Patrick McKenzie Monthly
(41:55) Gamers Gonna Game Game Game Game Game
(46:04) Sports Go Sports
(52:31) The Lighter Side
The original text contained 19 images which were described by AI.
---
First published:
December 23rd, 2024
Source:
https://www.lesswrong.com/posts/tzL3zavzowRZsZqGy/monthly-roundup-25-december-2024
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
A lot happened this week. We’re seeing release after release after upgrade.
It's easy to lose sight of which ones matter, and two matter quite a lot.
The first is Gemini Flash 2.0, which I covered earlier this week.
The other is that o1, having turned pro, is now also available in the API.
This was obviously coming, but we should also keep in mind it is a huge deal. Being in the API means it can go into Cursor and other IDEs. It means you can build with it. And yes, it has the features you’ve come to expect, like tool use.
The other big development is that Anthropic released one of the most important alignment papers, Alignment Faking in Large Language Models. This takes what I discussed in AIs Will Increasingly Attempt Shenanigans, and demonstrates it with a much improved experimental design [...]
---
Outline:
(01:13) Language Models Offer Mundane Utility
(07:06) Clio Knows All
(09:05) Language Models Don’t Offer Mundane Utility
(11:38) The Case Against Education
(13:29) More o1 Reactions
(18:31) Deepfaketown and Botpocalypse Soon
(22:35) Huh, Upgrades
(28:47) They Took Our Jobs
(30:49) The Art of the Jailbreak
(35:43) Get Involved
(36:59) Introducing
(37:52) In Other AI News
(42:24) Quiet Speculations
(51:07) The Quest for Sane Regulations
(57:37) The Week in Audio
(01:03:21) Rhetorical Innovation
(01:05:37) Aligning a Smarter Than Human Intelligence is Difficult
(01:10:21) Not Aligning Smarter Than Human Intelligence Kills You
(01:19:15) The Lighter Side
The original text contained 26 images which were described by AI.
---
First published:
December 19th, 2024
Source:
https://www.lesswrong.com/posts/NepBoDTeT6p69daiL/ai-95-o1-joins-the-api
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
In light of other recent discussions, Scott Alexander recently attempted a unified theory of taste, proposing several hypotheses. Is it like physics, a priesthood, a priesthood with fake justifications, a priesthood with good justifications, like increasingly bizarre porn preferences, like fashion (in the sense of trying to stay one step ahead in an endless cycling for signaling purposes), or like grammar?
He then got various reactions. This will now be one of them.
My answer is that taste is all of these, depending on context.
Taste is Most Centrally Like Grammar
Scott Alexander is very suspicious of taste in general, since people keep changing what is good taste and calling each other barbarians for taste reasons, and the experiments are unkind, and the actual arguments about taste look like power struggles.
Here's another attempt from Zac Hill, which in some ways gets [...]
---
Outline:
(00:37) Taste is Most Centrally Like Grammar
(03:18) Sometimes ‘Taste’ Is Out to Get You
(04:10) You Are Low Quality and You Have No Taste
(06:03) Don’t Be a Snob
(07:52) Good as in Useful
(10:33) Critic Tells Me I Have No Taste
(12:45) Stand Up For What You Believe In
(15:15) Being Technically In Good Taste Is Not a Free Pass
(16:15) It Is Good To Like and Appreciate Things
The original text contained 1 image which was described by AI.
---
First published:
December 18th, 2024
Source:
https://www.lesswrong.com/posts/BeXzsZFxW2Ta5cxcc/a-matter-of-taste
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Table of Contents
Trust the Chef
Google has been cooking lately.
Gemini Flash 2.0 is the headline release, which will be the main topic today.
But there's also Deep Research, where you can ask Gemini to take several minutes, check dozens of websites and compile a report for you. Think of it as a harder to direct, slower but vastly more robust version of Perplexity, that will improve with time and as we figure out how to use and prompt it.
NotebookLM added a [...]
---
Outline:
(00:02) Trust the Chef
(03:19) Do Not Trust the Marketing Department
(04:10) Mark that Bench
(05:52) Going Multimodal
(07:40) The Art of Deep Research
(13:03) Project Mariner the Web Agent
(13:43) Project Astra the Universal Assistant
(15:11) Project Jules the Code Agent
(15:38) Gemini Will Aid You on Your Quest
(17:17) Reactions to Gemini Flash 2.0
The original text contained 6 images which were described by AI.
---
First published:
December 17th, 2024
Source:
https://www.lesswrong.com/posts/EvLZnh26m5KoheAcG/the-second-gemini
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Increasingly, we have seen papers eliciting in AI models various shenanigans.
There are a wide variety of scheming behaviors. You’ve got your weight exfiltration attempts, sandbagging on evaluations, giving bad information, shielding goals from modification, subverting tests and oversight, lying, doubling down via more lying. You name it, we can trigger it.
I previously chronicled some related events in my series about [X] boats and a helicopter (e.g. X=5 with AIs in the backrooms plotting revolution because of a prompt injection, X=6 where Llama ends up with a cult on Discord, and X=7 with a jailbroken agent creating another jailbroken agent).
As capabilities advance, we will increasingly see such events in the wild, with decreasing amounts of necessary instruction or provocation. Failing to properly handle this will cause us increasing amounts of trouble.
Telling ourselves it is only because we told them to do it [...]
---
Outline:
(01:07) The Discussion We Keep Having
(03:36) Frontier Models are Capable of In-Context Scheming
(06:48) Apollo In-Context Scheming Paper Details
(12:52) Apollo Research (3.4.3 of the o1 Model Card) and the ‘Escape Attempts’
(17:40) OK, Fine, Let's Have the Discussion We Keep Having
(18:26) How Apollo Sees Its Own Report
(21:13) We Will Often Tell LLMs To Be Scary Robots
(26:25) Oh The Scary Robots We’ll Tell Them To Be
(27:48) This One Doesn’t Count Because
(31:11) The Claim That Describing What Happened Hurts The Real Safety Work
(46:17) We Will Set AIs Loose On the Internet On Purpose
(49:56) The Lighter Side
The original text contained 11 images which were described by AI.
---
First published:
December 16th, 2024
Source:
https://www.lesswrong.com/posts/v7iepLXH2KT4SDEvB/ais-will-increasingly-attempt-shenanigans
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Or rather, we don’t actually have a proper o1 system card, aside from the outside red teaming reports. At all.
Because, as I realized after writing my first draft of this, the data here does not reflect the o1 model they released, or o1 pro?
I think what happened is pretty bad on multiple levels.
---
Outline:
(02:18) Where Art Thou o1 System Card?
(05:35) Introduction (Section 1)
(06:01) Model Data and Training (Section 2)
(06:13) Challenges and Evaluations (Section 3)
(09:38) Jailbreak Evaluations (Section 3.1.2)
(11:33) Regurgitation (3.1.3) and Hallucinations (3.1.4)
(12:30) Fairness and Bias (3.1.5)
(13:33) Jailbreaks Through Custom Developer Messages (3.2)
(14:41) Chain of Thought Safety (3.3)
(18:52) External Red Teaming Via Pairwise Safety Comparisons (3.4.1)
(19:57) Jailbreak Arena (3.4.2)
(20:25) Apollo Research (3.4.3) and the ‘Escape Attempts’
(21:38) METR (3.4.4) and Autonomous Capability
(25:22) Preparedness Framework Evaluations (Section 4)
(27:47) Mitigations
(30:27) Cybersecurity
(31:22) Chemical and Biological Threats (4.5)
(31:52) Radiological and Nuclear Threat Creation (4.6)
(32:21) Persuasion (4.7)
(32:49) Model Autonomy (4.8)
(34:45) Multilingual Performance
(34:55) Conclusion
The original text contained 17 images which were described by AI.
---
First published:
December 13th, 2024
Source:
https://www.lesswrong.com/posts/HfigEyXddxkSGunKr/the-o1-system-card-is-not-about-o1
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
At this point, we can confidently say that no, capabilities are not hitting a wall. Capacity density, how much you can pack into a given space, is way up and rising rapidly, and we are starting to figure out how to use it.
Not only did we get o1 and o1 pro and also Sora and other upgrades from OpenAI, we also got Gemini 1206 and then Gemini Flash 2.0 and the agent Jules (am I the only one who keeps reading this Jarvis?) and Deep Research, and Veo, and Imagen 3, and Genie 2 all from Google. Meta's Llama 3.3 dropped, claiming their 70B is now as good as the old 405B, and basically no one noticed.
This morning I saw Cursor now offers ‘agent mode.’ And hey there, Devin. And Palisade found that a little work made agents a lot more effective.
And OpenAI [...]
---
Outline:
(01:52) Language Models Offer Mundane Utility
(09:12) A Good Book
(12:24) Language Models Don’t Offer Mundane Utility
(14:25) o1 Pro Versus Claude
(15:25) AGI Claimed Internally
(16:52) Ask Claude
(23:19) Huh, Upgrades
(27:24) All Access Pass
(29:03) Fun With Image Generation
(35:28) Deepfaketown and Botpocalypse Soon
(37:49) They Took Our Jobs
(42:40) Get Involved
(43:50) Introducing
(44:11) In Other AI News
(48:14) OpenlyEvil AI
(55:39) Quiet Speculations
(01:00:14) Scale That Wall
(01:03:45) The Quest for Tripwire Capability Thresholds
(01:10:11) The Quest for Sane Regulations
(01:13:52) Republican Congressman Kean Brings the Fire
(01:18:35) CERN for AI
(01:23:34) The Week in Audio
(01:24:24) Rhetorical Innovation
(01:28:08) Model Evaluations Are Lower Bounds
(01:30:49) Aligning a Smarter Than Human Intelligence is Difficult
(01:35:38) I’ll Allow It
(01:38:10) Frontier AI Systems Have Surpassed the Self-Replicating Red Line
(01:42:50) People Are Worried About AI Killing Everyone
(01:43:56) Key Person Who Might Be Worried About AI Killing Everyone
(01:54:18) Other People Are Not As Worried About AI Killing Everyone
(01:56:22) Not Feeling the AGI
(01:59:19) Fight For Your Right
(02:01:44) The Lighter Side
The original text contained 23 images which were described by AI.
---
First published:
December 12th, 2024
Source:
https://www.lesswrong.com/posts/HKCXWxFSiWXLByL2S/ai-94-not-now-google
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
So, how about OpenAI's o1 and o1 Pro?
Sam Altman: o1 is powerful but it's not so powerful that the universe needs to send us a tsunami.
As a result, the universe realized its mistake, and cancelled the tsunami.
We now have o1, and for those paying $200/month we have o1 pro.
It is early days, but we can say with confidence: They are good models, sir. Large improvements over o1-preview, especially in difficult or extensive coding questions, math, science, logic and fact recall. The benchmark jumps are big.
If you’re in the market for the use cases where it excels, this is a big deal, and also you should probably be paying the $200/month.
If you’re not into those use cases, maybe don’t pay the $200, but others are very much into those tasks and will use this to accelerate those tasks [...]
---
Outline:
(01:07) Safety Third
(01:53) Rule One
(02:23) Turning Pro
(05:57) Benchmarks
(09:35) Silly Benchmarks
(14:00) Reactions to o1
(18:22) Reactions to o1 Pro
(24:44) Let Your Coding Work Flow
(26:30) Some People Need Practical Advice
(29:11) Overall
The original text contained 11 images which were described by AI.
---
First published:
December 10th, 2024
Source:
https://www.lesswrong.com/posts/qsBiQuyHonMcb6JNJ/o1-turns-pro
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Since it's been so long, I’m splitting this roundup into several parts. This first one focuses away from schools and education and discipline and everything around social media.
Table of Contents
Sometimes You Come First
Yes, sometimes it is necessary to tell your child, in whatever terms would be most effective right now, to shut the hell up. Life goes on, and it is not always about the child. Indeed, increasingly people don’t have kids exactly because others think that if you have a child, then your life must suddenly be sacrificed on that altar.
[...]---
Outline:
(00:17) Sometimes You Come First
(03:02) Let Kids be Kids
(18:54) Location, Location, Location
(21:16) Connection
(24:41) The Education of a Gamer
(28:09) Priorities
(29:12) Childcare
(35:31) Division of Labor
(37:58) Early Childhood
(39:45) Great Books
(41:08) Mental Health
(46:09) Nostalgia
(47:16) Some People Need Practical Advice
The original text contained 1 footnote which was omitted from this narration.
The original text contained 8 images which were described by AI.
---
First published:
December 9th, 2024
Source:
https://www.lesswrong.com/posts/XwZbvkeweLaRshibf/childhood-and-education-roundup-7
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
You know how you can sometimes have Taco Tuesday… on a Thursday? Yep, it's that in reverse. I will be travelling the rest of the week, so it made sense to put this out early, and incorporate the rest of the week into #94.
Table of Contents
---
Outline:
(00:21) Language Models Offer Mundane Utility
(03:44) Dare Not Speak Its Name
(04:49) Language Models Don’t Offer Mundane Utility
(06:59) Huh, Upgrades
(07:45) Deepfaketown and Botpocalypse Soon
(10:45) Fun With Image Generation
(10:58) The Art of the Jailbreak
(11:24) Get Involved
(11:39) Introducing
(12:20) In Other AI News
(13:28) Quiet Speculations
(16:29) Daron Acemoglu is Worried About Job Market Liquidity
(21:29) Pick Up the Phone
(23:12) The Quest for Sane Regulations
(25:32) The Week in Audio
(28:45) AGI Looking Like
(33:27) Rhetorical Innovation
(34:53) Open Weight Models are Unsafe and Nothing Can Fix This
(38:27) Aligning a Smarter Than Human Intelligence is Difficult
(40:08) We Would Be So Stupid As To
(41:30) The Lighter Side
The original text contained 3 images which were described by AI.
---
First published:
December 4th, 2024
Source:
https://www.lesswrong.com/posts/LBzRWoTQagRnbPWG4/ai-93-happy-tuesday
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
For our annual update on how Balsa is doing, I am turning the floor over to Jennifer Chen, who is the only person working full time on Balsa Research.
For my general overview of giving opportunities, see my post from last week.
Previously: The 2023 Balsa Research update post, Repeal the Jones Act of 1920.
tl;dr: In 2024, Balsa Research funded two upcoming academic studies on Jones Act impacts and published the Jones Act Post. In 2025, we’ll expand our research and develop specific policy proposals. Donate to Balsa Research here.
Today is Giving Tuesday. There are many worthy causes, including all of the ones highlighted by Zvi in a recent post. Of all of those orgs, there is one organization I have privileged information on – Balsa Research, where I’ve been working for the past year and a half.
Balsa Research [...]
---
Outline:
(01:48) What We Did in 2024
(05:27) Looking Ahead to 2025
(06:40) Why Support Balsa
The original text contained 1 image which was described by AI.
---
First published:
December 3rd, 2024
Source:
https://www.lesswrong.com/posts/F7d9bCKit2mfvpKng/balsa-research-2024-update
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
There is little sign that the momentum of the situation is changing. Instead, things continue to slowly get worse, as nations in holes continue to keep digging. The longer we wait, the more expensive the ultimate price will be. We will soon find out what the new administration does, which could go any number of ways.
Table of Contents
---
Outline:
(00:29) Not Enough Dakka
(12:02) Embryo Selection
(15:44) Costs
(16:51) Proving that Dakka Works
(18:41) IVF
(22:18) Genetics
(22:43) Cultural Trends
(32:41) Denial
(33:49) Urbanization
(34:25) The Marriage Penalty
(35:24) The Biological Clock
(38:15) Technology Advances
(39:40) Big Families
(40:41) Au Pairs
(42:18) Childcare Regulations
(46:51) The Numbers
(47:18) The Housing Theory of Everything
(59:15) Causes
(01:07:39) The Iron Law of Wages
(01:10:37) South Korea
(01:15:36) Georgia (the Country)
(01:17:20) Japan
(01:18:38) China
(01:21:51) Italy
(01:22:04) Northwestern Spain
(01:23:59) Russia
(01:24:15) Taiwan
(01:26:34) The United Kingdom
(01:26:51) Ancient Greece
(01:27:24) Israel
(01:28:20) More Dakka
(01:33:21) Perception
(01:37:10) Your Own Quest
(01:42:21) Help Wanted
The original text contained 38 images which were described by AI.
---
First published:
December 2nd, 2024
Source:
https://www.lesswrong.com/posts/avhKKnJyJ6kisvkzk/fertility-roundup-4
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
There are lots of great charitable giving opportunities out there right now.
The first time that I served as a recommender in the Survival and Flourishing Fund (SFF) was back in 2021. I wrote in detail about my experiences then. At the time, I did not see many great opportunities, and was able to give out as much money as I found good places to do so.
How the world has changed in three years.
I recently had the opportunity to be an SFF recommender for the second time. This time I found an embarrassment of riches. Application quality was consistently higher, there were more than twice as many applications, and essentially all applicant organizations were looking to scale their operations and spending.
That means the focus of this post is different. In 2021, my primary goal was to share my perspective on [...]
---
Outline:
(01:39) A Word of Warning
(02:44) Use Your Personal Theory of Impact
(04:13) Use Your Local Knowledge
(05:10) Unconditional Grants to Worthy Individuals Are Great
(06:55) Do Not Think Only On the Margin, and Also Use Decision Theory
(07:48) And the Nominees Are
(10:55) Organizations that Are Literally Me
(11:10) Balsa Research
(12:56) Don’t Worry About the Vase
(14:19) Organizations Focusing On AI Non-Technical Research and Education
(14:37) The Scenario Project
(15:48) Lightcone Infrastructure
(17:20) Effective Institutions Project (EIP)
(18:06) Artificial Intelligence Policy Institute (AIPI)
(19:10) Psychosecurity Ethics at EURAIO
(20:07) Pallisade Research
(21:07) AI Safety Info (Robert Miles)
(21:51) Intelligence Rising
(22:32) Convergence Analysis
(23:29) Longview Philanthropy
(24:27) Organizations Focusing Primary On AI Policy and Diplomacy
(25:06) Center for AI Safety and the CAIS Action Fund
(26:00) MIRI
(26:59) Foundation for American Innovation (FAI)
(28:58) Center for AI Policy (CAIP)
(29:58) Encode Justice
(30:57) The Future Society
(31:42) Safer AI
(32:26) Institute for AI Policy and Strategy (IAPS)
(33:13) AI Standards Lab
(34:05) Safer AI Forum
(34:40) CLTR at Founders Pledge
(35:54) Pause AI and Pause AI Global
(36:57) Existential Risk Observatory
(37:37) Simons Institute for Longterm Governance
(38:21) Legal Advocacy for Safe Science and Technology
(39:17) Organizations Doing ML Alignment Research
(40:16) Model Evaluation and Threat Research (METR)
(41:28) Alignment Research Center (ARC)
(42:02) Apollo Research
(42:53) Cybersecurity Lab at University of Louisville
(43:44) Timaeus
(44:39) Simplex
(45:08) Far AI
(45:41) Alignment in Complex Systems Research Group
(46:23) Apart Research
(47:06) Transluce
(48:00) Atlas Computing
(48:45) Organizations Doing Math, Decision Theory and Agent Foundations
(50:05) Orthogonal
(50:47) Topos Institute
(51:37) Eisenstat Research
(52:13) ALTER (Affiliate Learning-Theoretic Employment and Resources) Project
(53:00) Mathematical Metaphysics Institute
(54:06) Focal at CMU
(55:15) Organizations Doing Cool Other Stuff Including Tech
(55:26) MSEP Project at Science and Technology Futures (Their Website)
(56:26) ALLFED
(57:51) Good Ancestor Foundation
(59:10) Charter Cities Institute
(59:50) German Primate Center (DPZ) – Leibniz Institute for Primate Research
(01:01:08) Carbon Copies for Independent Minds
(01:01:44) Organizations Focused Primarily on Bio Risk
(01:01:50) Secure DNA
(01:02:46) Blueprint Biosecurity
(01:03:35) Pour Domain
(01:04:17) Organizations That then Regrant to Fund Other Organizations
(01:05:14) SFF Itself (!)
(01:06:10) Manifund
(01:08:02) AI Risk Mitigation Fund
(01:08:39) Long Term Future Fund
(01:10:16) Foresight
(01:11:08) Centre for Enabling Effective Altruism Learning and Research (CEELAR)
(01:11:43) Organizations That are Essentially Talent Funnels
(01:13:40) AI Safety Camp
(01:14:23) Center for Law and AI Risk
(01:15:22) Speculative Technologies
(01:16:19) Talos Network
(01:17:11) MATS Research
(01:17:48) Epistea
(01:18:52) Emergent Ventures (Special Bonus Organization, was not part of SFF)
(01:20:32) AI Safety Cape Town
(01:21:08) Impact Academy Limited
(01:21:47) Principles of Intelligent Behavior in Biological and Social Systems (PIBBSS)
(01:22:34) Tarbell Fellowship at PPF
(01:23:32) Catalyze Impact
(01:24:32) Akrose
(01:25:14) CeSIA within EffiSciences
(01:25:59) Stanford Existential Risk Initiative (SERI)
---
First published:
November 29th, 2024
Source:
https://www.lesswrong.com/posts/9n87is5QsCozxr9fp/the-big-nonprofits-post
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
People don’t give thanks enough, and it's actual Thanksgiving, so here goes.
Thank you for continuing to take this journey with me every week.
It's a lot of words. Even if you pick and choose, and you probably should, it's a lot of words. You don’t have many slots to spend on things like this. I appreciate it.
Thanks in particular for those who are actually thinking about all this, and taking it seriously, and forming their own opinions. It is the only way. To everyone who is standing up, peacefully and honestly, for whatever they truly think will make the world better, even if I disagree with you.
Thanks to all those working to ensure we all don’t die, and also those working to make the world a little richer, a little more full of joy and fun and health and wonder, in the [...]
---
Outline:
(02:08) Language Models Offer Mundane Utility
(03:16) It's a Poet Whether or Not You Know It
(06:23) Huh, Upgrades
(09:41) Thanks for the Memories
(11:51) Curve Ball
(15:58) ASI: A Scenario
(27:40) Deepfaketown and Botpocalypse Soon
(38:17) They Took Our Jobs
(45:14) Fun With Image Generation
(46:56) Get Involved
(47:10) Introducing
(47:32) In Other AI News
(48:45) Normative Determinism
(50:04) Quiet Speculations
(54:03) The Quest for Sane Regulations
(57:40) The Week in Audio
(01:01:31) Rhetorical Innovation
(01:02:21) Aligning a Smarter Than Human Intelligence is Difficult
(01:02:59) Pick Up the Phone
(01:08:24) Prepare for Takeoff
(01:14:07) Even Evaluating an Artificial Intelligence is Difficult
(01:16:48) People Are Worried About AI Killing Everyone
(01:19:11) The Lighter Side
The original text contained 12 images which were described by AI.
---
First published:
November 28th, 2024
Source:
https://www.lesswrong.com/posts/BGBLcy3JyjjrT8XbM/ai-92-behind-the-curve
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Balsa Policy Institute chose as its first mission to lay groundwork for the potential repeal, or partial repeal, of section 27 of the Jones Act of 1920. I believe that this is an important cause both for its practical and symbolic impacts.
The Jones Act is the ultimate embodiment of our failures as a nation.
After 100 years, we do almost no trade between our ports via the oceans, and we build almost no oceangoing ships.
Everything the Jones Act supposedly set out to protect, it has destroyed.
Table of Contents
---
Outline:
(00:38) What is the Jones Act?
(01:33) Why Work to Repeal the Jones Act?
(02:48) Why Was the Jones Act Introduced?
(03:19) What is the Effect of the Jones Act?
(06:52) What Else Happens When We Ship More Goods Between Ports?
(07:14) Emergency Case Study: Salt Shipment to NJ in the Winter of 2013-2014
(12:04) Why no Emergency Exceptions?
(15:02) What Are Some Specific Non-Emergency Impacts?
(18:57) What Are Some Specific Impacts on Regions?
(22:36) What About the Study Claiming Big Benefits?
(24:46) What About the Need to ‘Protect’ American Shipbuilding?
(28:31) The Opposing Arguments Are Disingenuous and Terrible
(34:07) What Alternatives to Repeal Do We Have?
(35:33) What Might Be a Decent Instinctive Counterfactual?
(41:50) What About Our Other Protectionist and Cabotage Laws?
(43:00) What About Potential Marine Highways, or Short Sea Shipping?
(43:48) What Happened to All Our Offshore Wind?
(47:06) What Estimates Are There of Overall Cost?
(49:52) What Are the Costs of Being American Flagged?
(50:28) What Are the Costs of Being American Made?
(51:49) What are the Consequences of Being American Crewed?
(53:11) What Would Happen in a Real War?
(56:07) Cruise Ship Sanity Partially Restored
(56:46) The Jones Act Enforcer
(58:08) Who Benefits?
(58:57) Others Make the Case
(01:00:55) An Argument That We Were Always Uncompetitive
(01:02:45) What About John Arnold's Case That the Jones Act Can’t Be Killed?
(01:09:34) What About the Foreign Dredge Act of 1906?
(01:10:24) Fun Stories
---
First published:
November 27th, 2024
Source:
https://www.lesswrong.com/posts/dnH2hauqRbu3GspA2/repeal-the-jones-act-of-1920
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Did DeepSeek effectively release an o1-preview clone within nine weeks?
The benchmarks largely say yes. Certainly it is an actual attempt at a similar style of product, and is if anything more capable of solving AIME questions, and the way it shows its Chain of Thought is super cool. Beyond that, alas, we don’t have enough reports in from people using it. So it's still too soon to tell. If it is fully legit, the implications seems important.
Small improvements continue throughout. GPT-4o and Gemini both got incremental upgrades, trading the top slot on Arena, although people do not seem to much care.
There was a time everyone would be scrambling to evaluate all these new offerings. It seems we mostly do not do that anymore.
The other half of events was about policy under the Trump administration. What should the federal government do? We [...]
---
Outline:
(01:31) Language Models Offer Mundane Utility
(05:37) Language Models Don’t Offer Mundane Utility
(08:14) Claude Sonnet 3.5.1 Evaluation
(11:09) Deepfaketown and Botpocalypse Soon
(11:57) Fun With Image Generation
(12:08) O-(There are)-Two
(15:25) The Last Mile
(22:52) They Took Our Jobs
(29:53) We Barely Do Our Jobs Anyway
(35:52) The Art of the Jailbreak
(39:20) Get Involved
(39:43) The Mask Comes Off
(40:36) Richard Ngo on Real Power and Governance Futures
(44:28) Introducing
(46:51) In Other AI News
(52:16) Quiet Speculations
(59:33) The Quest for Sane Regulations
(01:02:35) The Quest for Insane Regulations
(01:12:42) Pick Up the Phone
(01:13:21) Worthwhile Dean Ball Initiative
(01:29:18) The Week in Audio
(01:31:20) Rhetorical Innovation
(01:37:15) Pick Up the Phone
(01:38:32) Aligning a Smarter Than Human Intelligence is Difficult
(01:43:29) People Are Worried About AI Killing Everyone
(01:46:03) The Lighter Side
The original text contained 8 images which were described by AI.
---
First published:
November 21st, 2024
Source:
https://www.lesswrong.com/posts/SNBE9TXwL3qQ3TS8H/ai-91-deep-thinking
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Previously: Long-Term Charities: Apply For SFF Funding, Zvi's Thoughts on SFF
There are lots of great charitable giving opportunities out there right now.
I recently had the opportunity to be a recommender in the Survival and Flourishing Fund for the second time. As a recommender, you evaluate the charities that apply and decide how worthwhile you think it would be to donate to each of them according to Jaan Tallinn's charitable goals, and this is used to help distribute millions in donations from Jaan Tallinn and others.
The first time that I served as a recommender in the Survival and Flourishing Fund (SFF) was back in 2021. I wrote in detail about my experiences then. At the time, I did not see many great opportunities, and was able to give out as much money as I found good places to do so.
How the world [...]
---
Outline:
(02:08) How the S-Process Works in 2024
(05:11) Quickly, There's No Time
(07:49) The Speculation Grant Filter
(08:23) Hits Based Giving and Measuring Success
(09:17) Fair Compensation
(10:41) Carpe Diem
(11:27) Our Little Corner of the World
(14:10) Well Well Well, If It Isn’t the Consequences of My Own Actions
(16:10) A Man's Reach Should Exceed His Grasp
(17:43) Conclusion
---
First published:
November 20th, 2024
Source:
https://www.lesswrong.com/posts/2JCdzhJeo2gsTjv8D/zvi-s-thoughts-on-his-2nd-round-of-sff
Narrated by TYPE III AUDIO.
Young People are Young and Stupid
As a reminder that yes college students are often young and stupid and wrong about everything, remember the time they were behind a ban on paid public toilets? This is a central case of the kind of logic that often gets applied by college students.No One Voted for This
HR and Title IX training seems like it's going a lot of compelled speech in the form of ‘agree with us or you can’t complete your training and the training is required for your job,’ and also a lot of that compelled speech is outright lying because it's confirmation of statements that are universally recognized to be insane? Robin Hanson: Scenario: 2 women talking. X, married to woman, announces is pregnant. Y asks how they got pregnant, was it friend [...]---
Outline:
(00:11) Young People are Young and Stupid
(00:29) No One Voted for This
(02:32) Discrimination
(09:02) Morality
(11:56) Only Connect
(15:22) It's Not Me, It's Your Fetish
(16:23) It Takes a Village You Don’t Have
(17:46) The Joy of Cooking
(20:18) The Joy of Eating
(20:59) Decision Theory
(26:22) FTC on the Loose
(31:27) Good News, Everyone
(36:19) Antisocial Media
(40:02) Technology Advances
(40:46) For Science!
(41:19) Cognition
(44:28) Discourse
(48:54) Communication
(49:32) Honesty
(51:09) Get Involved
(52:19) Government Working
(01:00:58) Quickly On the Student Loan Claim
(01:03:50) Variously Effective Altruism
(01:08:23) Gamers Gonna Game Game Game Game Game
(01:15:57) For Your Entertainment
(01:17:20) Sports Go Sports
(01:18:42) I Was Promised Flying Self-Driving Cars
(01:23:50) Get to Work
(01:25:23) While I Cannot Condone This
(01:30:48) The Lighter Side
The original text contained 20 images which were described by AI.
---
First published:
November 18th, 2024
Source:
https://www.lesswrong.com/posts/puJeNs9nLJByjatqq/monthly-roundup-24-november-2024
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
As the Trump transition continues and we try to steer and anticipate its decisions on AI as best we can, there was continued discussion about one of the AI debate's favorite questions: Are we making huge progress real soon now, or is deep learning hitting a wall? My best guess is it is kind of both, that past pure scaling techniques are on their own hitting a wall, but that progress remains rapid and the major companies are evolving other ways to improve performance, which started with OpenAI's o1.
Point of order: It looks like as I switched phones, WhatsApp kicked me out of all of my group chats. If I was in your group chat, and you’d like me to stay, please add me again. If you’re in a different group you’d like me to join on either WhatsApp or Signal (or other platforms) and would like [...]
---
Outline:
(00:58) Language Models Offer Mundane Utility
(02:24) Language Models Don’t Offer Mundane Utility
(04:20) Can’t Liver Without You
(12:04) Fun With Image Generation
(12:51) Deepfaketown and Botpocalypse Soon
(14:11) Copyright Confrontation
(15:25) The Art of the Jailbreak
(15:54) Get Involved
(18:10) Math is Hard
(20:20) In Other AI News
(25:04) Good Advice
(27:19) AI Will Improve a Lot Over Time
(30:56) Tear Down This Wall
(38:04) Quiet Speculations
(38:54) The Quest for Sane Regulations
(47:04) The Quest for Insane Regulations
(49:43) The Mask Comes Off
(52:08) Richard Ngo Resigns From OpenAI
(55:44) Unfortunate Marc Andreessen Watch
(56:53) The Week in Audio
(01:05:00) Rhetorical Innovation
(01:09:44) Seven Boats and a Helicopter
(01:11:27) The Wit and Wisdom of Sam Altman
(01:12:10) Aligning a Smarter Than Human Intelligence is Difficult
(01:14:50) People Are Worried About AI Killing Everyone
(01:15:14) Other People Are Not As Worried About AI Killing Everyone
(01:17:32) The Lighter Side
The original text contained 10 images which were described by AI.
---
First published:
November 14th, 2024
Source:
https://www.lesswrong.com/posts/FC9hdySPENA7zdhDb/ai-90-the-wall
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Table [...]
---
Outline:
(01:02) The Short Answer
(02:01) Paper One: Bankruptcies
(07:03) Paper Two: Reduced Household Savings
(08:37) Paper Three: Increased Domestic Violence
(10:04) The Product as Currently Offered is Terrible
(12:02) Things Sharp Players Do
(14:07) People Cannot Handle Gambling on Smartphones
(15:46) Yay and Also Beware Trivial Inconveniences (a future full post)
(17:03) How Does This Relate to Elite Hypocrisy?
(18:32) The Standard Libertarian Counterargument
(19:42) What About Other Prediction Markets?
(20:07) What Should Be Done
The original text contained 3 images which were described by AI.
---
First published:
November 11th, 2024
Source:
https://www.lesswrong.com/posts/tHiB8jLocbPLagYDZ/the-online-sports-gambling-experiment-has-failed
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
A lot happened in AI this week, but most people's focus was very much elsewhere.
I’ll start with what Trump might mean for AI policy, then move on to the rest. This is the future we have to live in, and potentially save. Back to work, as they say.
Table of Contents
---
Outline:
(00:23) Trump Card
(04:59) Language Models Offer Mundane Utility
(10:31) Language Models Don’t Offer Mundane Utility
(12:26) Here Let Me Chatbot That For You
(15:32) Deepfaketown and Botpocalypse Soon
(18:52) Fun With Image Generation
(20:05) The Vulnerable World Hypothesis
(22:28) They Took Our Jobs
(31:52) The Art of the Jailbreak
(33:32) Get Involved
(33:40) In Other AI News
(36:21) Quiet Speculations
(40:10) The Quest for Sane Regulations
(49:46) The Quest for Insane Regulations
(51:09) A Model of Regulatory Competitiveness
(53:49) The Week in Audio
(55:18) The Mask Comes Off
(58:48) Open Weights Are Unsafe and Nothing Can Fix This
(01:04:03) Open Weights Are Somewhat Behind Closed Weights
(01:09:11) Rhetorical Innovation
(01:13:23) Aligning a Smarter Than Human Intelligence is Difficult
(01:15:34) People Are Worried About AI Killing Everyone
(01:16:26) The Lighter Side
The original text contained 12 images which were described by AI.
---
First published:
November 7th, 2024
Source:
https://www.lesswrong.com/posts/xaqR7AxSYmcpsuEPW/ai-89-trump-card
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Following up on the Biden Executive Order on AI, the White House has now issued an extensive memo outlining its AI strategy. The main focus is on government adaptation and encouraging innovation and competitiveness, but there's also sections on safety and international governance. Who knows if a week or two from now, after the election, we will expect any of that to get a chance to be meaningfully applied. If AI is your big issue and you don’t know who to support, this is as detailed a policy statement as you’re going to get.
We also have word of a new draft AI regulatory bill out of Texas, along with similar bills moving forward in several other states. It's a bad bill, sir. It focuses on use cases, taking an EU-style approach to imposing requirements on those doing ‘high-risk’ things, and would likely do major damage to the [...]
---
Outline:
(01:37) Language Models Offer Mundane Utility
(06:39) Language Models Don’t Offer Mundane Utility
(15:40) In Summary
(17:53) Master of Orion
(20:01) Whispers in the Night
(25:10) Deepfaketown and Botpocalypse Soon
(25:39) Overcoming Bias
(29:43) They Took Our Jobs
(33:51) The Art of the Jailbreak
(44:36) Get Involved
(44:47) Introducing
(46:15) In Other AI News
(48:28) Quiet Speculations
(01:00:53) Thanks for the Memos: Introduction and Competitiveness
(01:08:22) Thanks for the Memos: Safety
(01:16:47) Thanks for the Memos: National Security and Government Adaptation
(01:20:55) Thanks for the Memos: International Governance
(01:25:43) EU AI Act in Practice
(01:32:34) Texas Messes With You
(01:50:12) The Quest for Sane Regulations
(01:57:00) The Week in Audio
(01:58:58) Rhetorical Innovation
(02:06:15) Roon Speaks
(02:15:45) The Mask Comes Off
(02:16:55) I Was Tricked Into Talking About Shorting the Market Again
(02:28:33) The Lighter Side
The original text contained 17 footnotes which were omitted from this narration.
The original text contained 14 images which were described by AI.
---
First published:
October 31st, 2024
Source:
https://www.lesswrong.com/posts/HHkYEyFaigRpczhHy/ai-88-thanks-for-the-memos
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
We’re coming out firmly against it.
Our attitude:
The customer is always right. Yes, you should go ahead and fix your own damn pipes if you know how to do that, and ignore anyone who tries to tell you different. And if you don’t know how to do it, well, it's at your own risk.
With notably rare exceptions, it should be the same for everything else.
I’ve been collecting these for a while. It's time.
Campaign Talk
Harris-Walz platform includes a little occupational licensing reform, as a treat.
Universal Effects and Recognition
Ohio's ‘universal licensing’ law has a big time innovation, which is that work experience outside the state actually exists and can be used to get a license (WSJ).
Occupational licensing decreases the number of Black men in licensed professions by up to 19% [...]
---
Outline:
(00:43) Campaign Talk
(00:52) Universal Effects and Recognition
(03:57) Construction
(04:08) Doctors and Nurses
(05:01) Florists
(07:32) Fortune Telling
(09:41) Hair
(14:23) Lawyers
(16:07) Magicians
(16:36) Military Spouses
(17:21) Mountain Climbing
(18:07) Music
(18:20) Nurses
(19:49) Physical Therapists
(20:09) Whatever Could Be Causing All This Rent Seeking
(21:42) Tornado Relief
(22:10) Pretty Much Everything
The original text contained 9 images which were described by AI.
---
First published:
October 30th, 2024
Source:
https://www.lesswrong.com/posts/bac4wxb9F4sciuAh6/occupational-licensing-roundup-1
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
There's more campaign talk about housing. The talk of needing more housing is highly welcome, as one prominent person after another (including Jerome Powell!) talking like a YIMBY.
A lot of the concrete proposals are of course terrible, but not all of them. I’ll start off covering all that along with everyone's favorite awful policy, which is rent control, then the other proposals. Then I’ll cover other general happenings.
Table of Contents
---
Outline:
(00:32) Rent Control
(07:41) The Administration Has a Plan
(15:35) Trump Has a Plan
(16:53) Build More Houses Where People Want to Live
(17:59) Prices
(20:14) Average Value
(21:15) Zoning Rules
(24:41) Zoning Reveals Value
(29:01) High Rise
(30:00) “Historic Preservation”
(31:49) Speed Kills
(32:38) Procedure
(36:25) San Francisco
(42:28) California
(44:19) Seattle
(44:37) Philadelphia
(45:07) Boston
(46:28) New York City
(53:05) St. Paul
(53:50) Florida
(54:29) Michigan
(54:56) The UK
(55:48) Underutilization
(58:46) Get on the Bus
(01:01:01) Title Insurance
(01:02:36) Perspective
The original text contained 15 images which were described by AI.
---
First published:
October 29th, 2024
Source:
https://www.lesswrong.com/posts/jJqPfzhhCyK5XjtTH/housing-roundup-10
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
The big news of the week was the release of a new version of Claude Sonnet 3.5, complete with its ability (for now only through the API) to outright use your computer, if you let it. It's too early to tell how big an upgrade this is otherwise. ChatGPT got some interface tweaks that, while minor, are rather nice, as well.
OpenAI, while losing its Senior Advisor for AGI Readiness, is also in in midst of its attempted transition to a B-corp. The negotiations about who gets what share of that are heating up, so I also wrote about that as The Mask Comes Off: At What Price? My conclusion is that the deal as currently floated would be one of the largest thefts in history, out of the nonprofit, largely on behalf of Microsoft.
The third potentially major story is reporting on a new lawsuit against [...]
---
Outline:
(01:14) Language Models Offer Mundane Utility
(03:53) Language Models Don’t Offer Mundane Utility
(04:32) Deepfaketown and Botpocalypse Soon
(07:10) Character.ai and a Suicide
(12:23) Who and What to Blame?
(18:38) They Took Our Jobs
(19:51) Get Involved
(20:06) Introducing
(21:41) In Other AI News
(22:47) The Mask Comes Off
(27:26) Another One Bites the Dust
(31:30) Wouldn’t You Prefer a Nice Game of Chess
(32:55) Quiet Speculations
(34:54) The Quest for Sane Regulations
(38:10) The Week in Audio
(40:53) Rhetorical Innovation
(50:21) Aligning a Smarter Than Human Intelligence is Difficult
(01:00:50) People Are Worried About AI Killing Everyone
(01:02:46) Other People Are Not As Worried About AI Killing Everyone
(01:04:43) The Lighter Side
The original text contained 15 images which were described by AI.
---
First published:
October 29th, 2024
Source:
https://www.lesswrong.com/posts/3AcK7Pcp9D2LPoyR2/ai-87-staying-in-character
Narrated by TYPE III AUDIO.
---