LessWrong (30+ Karma)

Epistemic status: Noticing confusion

There is little discussion happening on LessWrong with regards to AI governance and outreach. Meanwhile, these efforts could buy us time to figure out technical alignment. And even if we figure out technical alignment, we still have to solve crucial governmental challenges so that totalitarian lock-in or gradual disempowerment don't become the default outcome of deploying aligned AGI.

Here's three reasons why we think we might want to shift much more resources towards governance and outreach:

1. MIRI's shift in strategy

The Machine Intelligence Research Institute (MIRI), traditionally focused on technical alignment research, has pivoted to broader outreach. They write in their 2024 end of year update:

Although we continue to support some AI alignment research efforts, we now believe that absent an international government effort to suspend frontier AI research, an extinction-level catastrophe is extremely likely.

As a consequence, our new focus is [...]

---

Outline:

(00:45) 1. MIRIs shift in strategy

(01:34) 2. Even if we solve technical alignment, Gradual Disempowerment seems to make catastrophe the default outcome

(02:52) 3. We have evidence that the governance naysayers are badly calibrated

(03:33) Conclusion

---

First published:
April 12th, 2025

Source:
https://www.lesswrong.com/posts/Y49znC2qfL9SXkT7e/why-does-lw-not-put-much-more-focus-on-ai-governance-and

Narrated by TYPE III AUDIO.

In this post I lay out a concrete vision of how reward-seekers and schemers might function. I describe the relationship between higher level goals, explicit reasoning, and learned heuristics. I explain why I expect reward-seekers and schemers to dominate proxy-aligned models given sufficiently rich training environments (and sufficient reasoning ability).

A key point is that explicit reward seekers can still contain large quantities of learned heuristics (context-specific drives). By viewing these drives as instrumental and having good instincts for when to trust them, a reward seeker can capture the benefits of both instinctive adaptation and explicit reasoning without paying much of a speed penalty.

Core claims

1. To get high reward, models must learn context-specific heuristics which are not derivable by reasoning from the goal of reward maximization.[1]
2. It is also true that thinking explicitly about reward is sometimes a waste of time and punished by speed penalties.
3. It [...]

---

Outline:

(00:52) Core claims

(01:52) Characterizing reward-seekers

(06:59) When will models think about reward?

(10:12) What I expect schemers to look like

(12:43) What will terminal reward seekers do off-distribution?

(13:24) What factors affect the likelihood of scheming and/or terminal reward seeking?

(14:09) What about CoT models?

(14:42) Relationship of subgoals to their superior goals

(16:19) A story about goal reflection

(18:54) Thoughts on compression

(21:30) Appendix: Distribution over worlds

(24:44) Canary string

(25:01) Acknowledgements

The original text contained 6 footnotes which were omitted from this narration.

---

First published:
April 11th, 2025

Source:
https://www.lesswrong.com/posts/ntDA4Q7BaYhWPgzuq/reward-seekers

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Cross-posted from Substack.

AI job displacement will affect young people first, disrupting the usual succession of power and locking the next generation out of our institutions. I’m coining the term youth lockout to describe this phenomenon.

Youth Lockout

We are on track to build AI agents that can independently perform valuable intellectual labour. These agents will directly compete with human workers for roles in the labour market, often offering services at lower cost and greater speeds.

In historical cases of automation, such as the industrial revolution, automation reduced the number of human jobs in some industries but created enough new opportunities that overall demand for labour went up. AI automation will be very different from these examples because AI is a much more general-purpose technology. With time, AI will likely perform all economic tasks at human or superhuman levels, meaning any new firm or industry will be able to [...]

---

Outline:

(00:24) Youth Lockout

(03:34) Impacts

(06:54) Conclusion

---

First published:
April 11th, 2025

Source:
https://www.lesswrong.com/posts/tWuYhdajaXx4WzMHz/youth-lockout

Narrated by TYPE III AUDIO.

Paper is good. Somehow, a blank page and a pen makes the universe open up before you. Why paper has this unique power is a mystery to me, but I think we should all stop trying to resist this reality and just accept it.

Also, the world needs way more mundane blogging.

So let me offer a few observations about paper. These all seem quite obvious. But it took me years to find them, and they’ve led me to a non-traditional lifestyle, paper-wise.

Observation 1: The primary value of paper is to facilitate thinking.

For a huge percentage of tasks that involve thinking, getting some paper and writing / drawing / scribbling on it makes the task easier. I think most people agree with that. So why don’t we act on it? If paper came as a pill, everyone would take it. Paper, somehow, is underrated.

But note, paper [...]

---

Outline:

(00:38) Observation 1: The primary value of paper is to facilitate thinking.

(01:23) Observation 2: If you don't have a system, you won't get much benefit from paper.

(02:02) Observation 3: User experience matters.

(02:52) Observation 4: Categorization is hard.

(03:35) Paper systems I've used

---

First published:
April 11th, 2025

Source:
https://www.lesswrong.com/posts/RdHEhPKJG6mp39Agw/paper

Narrated by TYPE III AUDIO.

Summary

OpenAI recently released the Responses API. Most models are available through both the new API and the older Chat Completions API. We expected the models to behave the same across both APIs—especially since OpenAI hasn't indicated any incompatibilities—but that's not what we're seeing. In fact, in some cases, the differences are substantial. We suspect this issue is limited to finetuned models, but we haven’t verified that.

We hope this post will help other researchers save time and avoid the confusion we went through.

Key takeaways are that if you're using finetuned models:

• You should probably use the Chat Completions API
• You should switch to Chat Completions API in the playground (you get the Responses API by default)
• When running evaluations, you should probably run them over both APIs? It's hard to say what is the "ground truth" here.

Example: ungrammatical model

In one of our [...]

---

Outline:

(01:11) Example: ungrammatical model

(02:08) Ungrammatical model is not the only one

(02:37) Whats going on?

The original text contained 1 footnote which was omitted from this narration.

---

First published:
April 11th, 2025

Source:
https://www.lesswrong.com/posts/vTvPvCH2G9cbcFY8a/openai-responses-api-changes-models-behavior

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It's generally agreed that as AIs get more capable, risks from misalignment increase. But there are a few different mechanisms by which more capable models are riskier, and distinguishing between those mechanisms is important when estimating the misalignment risk posed at a particular level of capabilities or by a particular model.

There are broadly 3 reasons why misalignment risks increase with capabilities:

• As AIs get more capable, the chance of egregious misalignment increases: AIs with more powerful opaque reasoning abilities are more likely to become egregiously misaligned—sufficiently misaligned that they strategically deceive and work against their developers—without this being detected or trained out. We should expect opaque reasoning abilities generally increase as models get more capable, though it isn't entirely obvious what the relationship between opaque and non-opaque cognition will be in future AI systems. Additionally, I expect that as AIs get more capable their capabilities [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
April 11th, 2025

Source:
https://www.lesswrong.com/posts/NDotm7oLHfR56g4sD/why-do-misalignment-risks-increase-as-ais-get-more-capable

Narrated by TYPE III AUDIO.

Google Lays Out Its Safety Plans

I want to start off by reiterating kudos to Google for actually laying out its safety plan. No matter how good the plan, it's much better to write down and share the plan than it is to not share the plan, which in turn is much better than not having a formal plan.

They offer us a blog post, a full monster 145 page paper (so big you have to use Gemini!) and start off the paper with a 10 page summary.

The full paper is full of detail about what they think and plan, why they think and plan it, answers to objections and robust discussions. I can offer critiques, but I couldn’t have produced this document in any sane amount of time, and I will be skipping over a lot of interesting things in the full paper because [...]

---

Outline:

(00:58) Core Assumptions

(08:14) The Four Questions

(11:46) Taking This Seriously

(16:04) A Problem For Future Earth

(17:53) That's Not My Department

(23:22) Terms of Misuse

(27:50) Misaligned!

(36:17) Aligning a Smarter Than Human Intelligence is Difficult

(51:48) What Is The Goal?

(54:43) Have You Tried Not Trying?

(58:07) Put It To The Test

(59:44) Mistakes Will Be Made

(01:00:35) Then You Have Structural Risks

---

First published:
April 11th, 2025

Source:
https://www.lesswrong.com/posts/hvEikwtsbf6zaXG2s/on-google-s-safety-plan

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Authors: Eli Lifland, Nikola Jurkovic[1], FutureSearch[2]

This is supporting research for AI 2027. We'll be cross-posting these over the next week or so.

Assumes no large-scale catastrophes happen (e.g., a solar flare, a pandemic, nuclear war), no government or self-imposed slowdown, and no significant supply chain disruptions. All forecasts give a substantial chance of superhuman coding arriving in 2027.

Summary

We forecast when the leading AGI company will internally develop a superhuman coder (SC): an AI system that can do any coding tasks that the best AGI company engineer does, while being much faster and cheaper. At this point, the SC will likely speed up AI progress substantially as is explored in our takeoff forecast.

We first show Method 1: time-horizon-extension, a relatively simple model which forecasts when SC will arrive by extending the trend established by METR's report of AIs accomplishing tasks that take humans increasing amounts [...]

---

Outline:

(00:56) Summary

(02:43) Defining a superhuman coder (SC)

(03:35) Method 1: Time horizon extension

(05:05) METR's time horizon report

(06:30) Forecasting SC's arrival

(06:54) Method 2: Benchmarks and gaps

(06:59) Time to RE-Bench saturation

(07:03) Why RE-Bench?

(09:25) Forecasting saturation via extrapolation

(12:42) AI progress speedups after saturation

(14:04) Time to cross gaps between RE-Bench saturation and SC

(14:32) What are the gaps in task difficulty between RE-Bench saturation and SC?

(15:11) Methodology

(17:25) How fast can the task difficulty gaps be crossed?

(23:31) Other factors for benchmarks and gaps

(23:46) Compute scaling and algorithmic progress slowdown

(24:43) Gap between internal and external deployment

(25:20) Intermediate speedups

(26:55) Overall benchmarks and gaps forecasts

(27:44) Appendix

(27:47) Individual Forecaster Views for Benchmark-Gap Model Factors

(27:53) Engineering complexity: handling complex codebases

(31:16) Feedback loops: Working without externally provided feedback

(37:28) Parallel projects: Handling several interacting projects

(38:45) Specialization: Specializing in skills specific to frontier AI development

(40:17) Cost and speed

(48:48) Other task difficulty gaps

(50:52) Superhuman Coder (SC): time horizon and reliability requirements

(55:53) RE-Bench saturation resolution criteria

The original text contained 19 footnotes which were omitted from this narration.

---

First published:
April 10th, 2025

Source:
https://www.lesswrong.com/posts/ggqSg7bSLChanfunf/forecasting-time-to-automated-superhuman-coders-ai-2027

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: Briefer and more to the point than my model of what is going on with LLMs, but also lower effort.

Here is the paper. The main reaction I am talking about is AI 2027, but also basically every lesswrong take on AI 2027.

A lot of people believe in very short AI timelines, say <2030. They want to justify this with some type of outside view, straight-lines-on-graphs argument, which is pretty much all we've got because nobody has a good inside view on deep learning (change my mind).

The outside view, insofar is that is a well-defined thing, does not justify very short timelines.

If AGI were arriving in 2030, the outside view says interest rates would be very high (I'm not particularly knowledgeable about this and might have the details wrong but see the analysis here, I believe the situation is still [...]

---

First published:
April 10th, 2025

Source:
https://www.lesswrong.com/posts/BrHv7wc6hiJEgJzHW/reactions-to-metr-task-length-paper-are-insane

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

When I was a really small kid, one of my favorite activities was to try and dam up the creek in my backyard. I would carefully move rocks into high walls, pile up leaves, or try patching the holes with sand. The goal was just to see how high I could get the lake, knowing that if I plugged every hole, eventually the water would always rise and defeat my efforts. Beaver behaviour.

One day, I had the realization that there was a simpler approach. I could just go get a big 5 foot long shovel, and instead of intricately locking together rocks and leaves and sticks, I could collapse the sides of the riverbank down and really build a proper big dam. I went to ask my dad for the shovel to try this out, and he told me, very heavily paraphrasing, 'Congratulations. You've [...]

---

First published:
April 10th, 2025

Source:
https://www.lesswrong.com/posts/rLucLvwKoLdHSBTAn/playing-in-the-creek

Linkpost URL:
https://hgreer.com/PlayingInTheCreek

Narrated by TYPE III AUDIO.

When complex systems fail, it is often because they have succumbed to what we call "disempowerment spirals" — self-reinforcing feedback loops where an initial threat progressively undermines the system's capacity to respond, leading to accelerating vulnerability and potential collapse.

Consider a city gradually falling under the control of organized crime. The criminal organization doesn't simply overpower existing institutions through sheer force. Rather, it systematically weakens the city's response mechanisms: intimidating witnesses, corrupting law enforcement, and cultivating a reputation that silences opposition. With each incremental weakening of response capacity, the criminal faction acquires more power to further dismantle resistance, creating a downward spiral that can eventually reach a point of no return.

This basic pattern appears across many different domains and scales:

• HIV progressively destroys the immune system designed to fight it.
• Anxiety, burnout, or depression deplete executive function, which is required for taking steps to address [...]

---

Outline:

(02:49) Common Themes

(02:52) Three Types of Response Capacity

(04:31) Broad Disempowerment

(05:12) Polycrises

(06:06) Critical Threshold

(07:47) Disempowerment spirals and existential risk

The original text contained 1 footnote which was omitted from this narration.

---

First published:
April 10th, 2025

Source:
https://www.lesswrong.com/posts/KtBXDakpwy6myBrKd/disempowerment-spirals-as-a-likely-mechanism-for-existential

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In recent months, the CEOs of leading AI companies have grown increasingly confident about rapid progress:

• OpenAI's Sam Altman: Shifted from saying in November “the rate of progress continues” to declaring in January “we are now confident we know how to build AGI”

• Anthropic's Dario Amodei: Stated in January “I’m more confident than I’ve ever been that we’re close to powerful capabilities… in the next 2-3 years”

• Google DeepMind's Demis Hassabis: Changed from “as soon as 10 years” in autumn to “probably three to five years away” by January.

What explains the shift? Is it just hype? Or could we really have Artificial General [...]

---

Outline:

(04:12) In a nutshell

(05:54) I. What's driven recent AI progress? And will it continue?

(06:00) The deep learning era

(08:33) What's coming up

(09:35) 1. Scaling pretraining to create base models with basic intelligence

(09:42) Pretraining compute

(13:14) Algorithmic efficiency

(15:08) How much further can pretraining scale?

(16:58) 2. Post training of reasoning models with reinforcement learning

(22:50) How far can scaling reasoning models continue?

(26:09) 3. Increasing how long models think

(29:04) 4. The next stage: building better agents

(34:59) How far can the trend of improving agents continue?

(37:15) II. How good will AI become by 2030?

(37:20) The four drivers projected forwards

(39:01) Trend extrapolation of AI capabilities

(40:19) What jobs would these systems be able to help with?

(41:11) Software engineering

(42:34) Scientific research

(43:45) AI research

(44:57) What's the case against impressive AI progress by 2030?

(49:39) When do the 'experts' expect AGI to arrive?

(51:04) III. Why the next 5 years are crucial

(52:07) Bottlenecks around 2030

(55:49) Two potential futures for AI

(57:52) Conclusion

(59:08) Use your career to tackle this issue

(59:32) Further reading

The original text contained 47 footnotes which were omitted from this narration.

---

First published:
April 9th, 2025

Source:
https://www.lesswrong.com/posts/NkwHxQ67MMXNqRnsR/the-case-for-agi-by-2030

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Diffractor is the first author of this paper.
Official title: "Regret Bounds for Robust Online Decision Making"

Abstract: We propose a framework which generalizes "decision making with structured observations" by allowing robust (i.e. multivalued) models. In this framework, each model associates each decision with a convex set of probability distributions over outcomes. Nature can choose distributions out of this set in an arbitrary (adversarial) manner, that can be nonoblivious and depend on past history. The resulting framework offers much greater generality than classical bandits and reinforcement learning, since the realizability assumption becomes much weaker and more realistic. We then derive a theory of regret bounds for this framework. Although our lower and upper bounds are not tight, they are sufficient to fully characterize power-law learnability. We demonstrate this theory in two special cases: robust linear bandits and tabular robust online reinforcement learning. In both cases [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
April 10th, 2025

Source:
https://www.lesswrong.com/posts/LgLez8aeK24PbyyQJ/new-paper-infra-bayesian-decision-estimation-theory

Linkpost URL:
https://arxiv.org/abs/2504.06820

Narrated by TYPE III AUDIO.

This is part of the MIRI Single Author Series. Pieces in this series represent the beliefs and opinions of their named authors, and do not claim to speak for all of MIRI.

Okay, I'm annoyed at people covering AI 2027 burying the lede, so I'm going to try not to do that. The authors predict a strong chance that all humans will be (effectively) dead in 6 years, and this agrees with my best guess about the future. (My modal timeline has loss of control of Earth mostly happening in 2028, rather than late 2027, but nitpicking at that scale hardly matters.) Their timeline to transformative AI also seems pretty close to the perspective of frontier lab CEO's (at least Dario Amodei, and probably Sam Altman) and the aggregate market opinion of both Metaculus and Manifold!

If you look on those market platforms you get graphs like this:

Both [...]

---

Outline:

(02:23) Mode ≠ Median

(04:50) Theres a Decent Chance of Having Decades

(06:44) More Thoughts

(08:55) Mid 2025

(09:01) Late 2025

(10:42) Early 2026

(11:18) Mid 2026

(12:58) Late 2026

(13:04) January 2027

(13:26) February 2027

(14:53) March 2027

(16:32) April 2027

(16:50) May 2027

(18:41) June 2027

(19:03) July 2027

(20:27) August 2027

(22:45) September 2027

(24:37) October 2027

(26:14) November 2027 (Race)

(29:08) December 2027 (Race)

(30:53) 2028 and Beyond (Race)

(34:42) Thoughts on Slowdown

(38:27) Final Thoughts

---

First published:
April 9th, 2025

Source:
https://www.lesswrong.com/posts/Yzcb5mQ7iq4DFfXHx/thoughts-on-ai-2027

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Timothy and I have recorded a new episode of our podcast with Austin Chen of Manifund (formerly of Manifold, behind the scenes at Manifest).

The start of the conversation was contrasting each of our North Stars- Winning (Austin), Truthseeking (me), and Flow (Timothy), but I think the actual theme might be “what is an acceptable amount of risk taking?” We eventually got into a discussion of Sam Bankman-Fried, where Austin very bravely shared his position that SBF has been unwisely demonized and should be “freed and put back to work”. He by no means convinced me or Timothy of this, but I deeply appreciate the chance for a public debate.

Episode:

Transcript (this time with filler words removed by AI)

Editing policy: we allow guests (and hosts) to redact things they said, on the theory that this is no worse than not saying them [...]

---

First published:
April 7th, 2025

Source:
https://www.lesswrong.com/posts/Xuw6vmnQv6mSMtmzR/austin-chen-on-winning-risk-taking-and-ftx

Narrated by TYPE III AUDIO.

Short AI takeoff timelines seem to leave no time for some lines of alignment research to become impactful. But any research rebalances the mix of currently legible research directions that could be handed off to AI-assisted alignment researchers or early autonomous AI researchers whenever they show up. So even hopelessly incomplete research agendas could still be used to prompt future capable AI to focus on them, while in the absence of such incomplete research agendas we'd need to rely on AI's judgment more completely. This doesn't crucially depend on giving significant probability to long AI takeoff timelines, or on expected value in such scenarios driving the priorities.

Potential for AI to take up the torch makes it reasonable to still prioritize things that have no hope at all of becoming practical for decades (with human effort). How well AIs can be directed to advance a line of research [...]

---

First published:
April 9th, 2025

Source:
https://www.lesswrong.com/posts/3NdpbA6M5AM2gHvTW/short-timelines-don-t-devalue-long-horizon-research

Narrated by TYPE III AUDIO.

Researchers used RNA sequencing to observe how cell types change during brain development. Other researchers looked at connection patterns of neurons in brains. Clear distinctions have been found between all mammals and all birds. They've concluded intelligence developed independently in birds and mammals; I agree. This is evidence for convergence of general intelligence.

---

First published:
April 8th, 2025

Source:
https://www.lesswrong.com/posts/Z2FfGJh2gAA6EXezp/birds-and-mammals-independently-evolved-intelligence

Linkpost URL:
https://www.quantamagazine.org/intelligence-evolved-at-least-twice-in-vertebrate-animals-20250407/

Narrated by TYPE III AUDIO.

The first AI war will be in your computer and/or smartphone.

Companies want to get customers / users. The ones more willing to take "no" for an answer will lose in the competition. You don't need a salesman when an install script (ideally, run without the user's consent) does a better job; and most users won't care.

Sometimes Windows during a system update removes dual boot from my computer and replaces it with Windows-only boot. Sometimes a web browser tells me "I noticed that I am not your default browser, do you want me to register as your default browser?" Sometimes Windows during a system update just registers current Microsoft's browser as a default browser without asking. At least this is what used to happen in the past.

I expect similar dynamics with AIs, soon. The companies will push really hard to make you use their AI. The smaller [...]

---

First published:
April 8th, 2025

Source:
https://www.lesswrong.com/posts/EEv4w57wMcrpn4vvX/the-first-ai-war-will-be-in-your-computer

Narrated by TYPE III AUDIO.

If there turns out not to be an AI crash, you get a 1/(1+7) * $25,000 = $3,125
If there is an AI crash, you transfer $25k to me.

If you believe that AI is going to keep getting more capable, pushing rapid user growth and work automation across sectors, this is near free money. But to be honest, I think there will likely be an AI crash in the next 5 years, and on average expect to profit well from this one-year bet.

If I win though, I want to give the $25k to organisers who can act fast to restrict the weakened AI corps in the wake of the crash. So bet me if you're highly confident that you'll win or just want to hedge the community against the possibility of a crash.

To make this bet, we need to set the threshold for a market [...]

---

First published:
April 8th, 2025

Source:
https://www.lesswrong.com/posts/4o3AkH7j8XbppF7TG/who-wants-to-bet-me-usd25k-at-1-7-odds-that-there-won-t-be

Narrated by TYPE III AUDIO.

In this post, we present a replication and extension of an alignment faking model organism:

• Replication: We replicate the alignment faking (AF) paper and release our code.
• Classifier Improvements: We significantly improve the precision and recall of the AF classifier. We release a dataset of ~100 human-labelled examples of AF for which our classifier achieves an AUROC of 0.9 compared to 0.6 from the original classifier.
• Evaluating More Models: We find Llama family models, other open source models, and GPT-4o do not AF in the prompted-only setting when evaluating using our new classifier (other than a single instance with Llama 3 405B).
• Extending SFT Experiments: We run supervised fine-tuning (SFT) experiments on Llama (and GPT4o) and find that AF rate increases with scale. We release the fine-tuned models on Huggingface and scripts.
• Alignment faking on 70B: We find that Llama 70B alignment fakes when both using the system prompt in the [...]

---

Outline:

(02:43) Method

(02:46) Overview of the Alignment Faking Setup

(04:22) Our Setup

(06:02) Results

(06:05) Improving Alignment Faking Classification

(10:56) Replication of Prompted Experiments

(14:02) Prompted Experiments on More Models

(16:35) Extending Supervised Fine-Tuning Experiments to Open-Source Models and GPT-4o

(23:13) Next Steps

(25:02) Appendix

(25:05) Appendix A: Classifying alignment faking

(25:17) Criteria in more depth

(27:40) False positives example 1 from the old classifier

(30:11) False positives example 2 from the old classifier

(32:06) False negative example 1 from the old classifier

(35:00) False negative example 2 from the old classifier

(36:56) Appendix B: Classifier ROC on other models

(37:24) Appendix C: User prompt suffix ablation

(40:24) Appendix D: Longer training of baseline docs

The original text contained 1 footnote which was omitted from this narration.

---

First published:
April 8th, 2025

Source:
https://www.lesswrong.com/posts/Fr4QsQT52RFKHvCAH/alignment-faking-revisited-improved-classifiers-and-open

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Kevin Roose in The New York Times

---

Outline:

(00:21) Kevin Roose in The New York Times

(02:56) Eli Lifland Offers Takeaways

(04:23) Scott Alexander Offers Takeaways

(05:34) Others Takes on Scenario 2027

(05:39) Having a Concrete Scenario is Helpful

(08:37) Writing It Down Is Valuable Even If It Is Wrong

(10:00) Saffron Huang Worries About Self-Fulfilling Prophecy

(18:18) Phillip Tetlock Calibrates His Skepticism

(21:38) Jan Kulveit Wants to Bet

(23:08) Matthew Barnett Debates How To Evaluate the Results

(24:38) Teortaxes for China and Open Models and My Response

(31:53) Others Wonder About PRC Passivity

(33:40) Timothy Lee Remains Skeptical

(35:16) David Shapiro for the Accelerationists and Scott's Response

(45:29) LessWrong Weighs In

(46:59) Other Reactions

(50:05) Next Steps

(52:34) The Lighter Side

---

First published:
April 8th, 2025

Source:
https://www.lesswrong.com/posts/gyT8sYdXch5RWdpjx/ai-2027-responses

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Spoiler: “So after removing the international students from the calculations, and using the middle-of-the-range estimates, the conclusion: The top-scoring 19,000 American students each year are competing in top-20 admissions for about 12,000 spots out of 44,000 total. Among the Ivy League + MIT + Stanford, they’re competing for about 6,500 out of 15,800 total spots.”

It's well known that

1. Admission to top universities is very competitive in America and even top SAT scores (1550+ out of 1600) paired with a 4.0 GPA doesn’t guarantee admission to a top school.
2. Even top universities take into account race-based affirmative action, athletic recruitment, “Dean's Interest List”-type tracking systems for children of donors or notable persons, and legacy preference for children of alumni.

But many people are under the misconception that the resulting “rat race”—the highly competitive and strenuous admissions ordeal—is the inevitable result of the limited class sizes among top [...]

---

First published:
April 7th, 2025

Source:
https://www.lesswrong.com/posts/vptDgKbiEwsKAFuco/american-college-admissions-doesn-t-need-to-be-so

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

My thoughts on the recently posted story.

Caveats

• I think it's great that the AI Futures Project wrote up a detailed scenario.
• I enjoy it.
• Every part of the story i didn't comment on here is either fine or excellent.
• This is one of the most realistic scenarios i've read.
• All detailed predictions contain errors.
• The authors of this scenario don't claim it's the most likely future.
• If the speed of 2018-2025 was the typical rate of progress in software, then AI 2027 would be realistic.

Core Disagreements

• Early 2026: OpenBrain is making algorithmic progress 50% faster.
  • As with many parts of this scenario, i think this is plausible but too fast. 150% productivity is a lot in business terms, & the scenario doesn't provide much detail for why this is 150% as opposed to 110%. In my software development experience, organizations are bottlenecked by their [...]

---

Outline:

(00:14) Caveats

(00:52) Core Disagreements

(05:33) Minor Details

(12:04) Overall

---

First published:
April 5th, 2025

Source:
https://www.lesswrong.com/posts/6Aq2FBZreyjBp6FDt/most-questionable-details-in-ai-2027

Narrated by TYPE III AUDIO.

Daniel Kokotajlo has launched AI 2027, Scott Alexander introduces it here. AI 2027 is a serious attempt to write down what the future holds. His ‘What 2026 Looks Like’ was very concrete and specific, and has proved remarkably accurate given the difficulty level of such predictions.

I’ve had the opportunity to play the wargame version of the scenario described in 2027, and I reviewed the website prior to publication and offered some minor notes. Whenever I refer to a ‘scenario’ in this post I’m talking about Scenario 2027.

There's tons of detail here. The research here, and the supporting evidence and citations and explanations, blow everything out of the water. It's vastly more than we usually see, and dramatically different from saying ‘oh I expect AGI in 2027’ or giving a timeline number. This lets us look at what happens in concrete detail, figure out where we [...]

---

Outline:

(02:00) The Structure of These Post

(03:37) Coverage of the Podcast

---

First published:
April 7th, 2025

Source:
https://www.lesswrong.com/posts/vnkH6JrGu2AxtDTyu/ai-2027-dwarkesh-s-podcast-with-daniel-kokotajlo-and-scott

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The catholic church has always had a complicated relationship with homosexuality.

The central claim of Frederic Martel's 2019 book In the Closet of the Vatican is that the majority of the church's leadership in Rome are semi-closeted homosexuals, or more colorfully, "homophiles".

So the omnipresence of homosexuals in the Vatican isn’t just a matter of a few black sheep, or the ‘net that caught the bad fish’, as Josef Ratzinger put it. It isn’t a ‘lobby’ or a dissident movement; neither is it a sect of Freemasonry inside the holy see: it's a system. It isn’t a tiny minority; it's a big majority.

At this point in the conversation, I ask Francesco Lepore to estimate the size of this community, all tendencies included.

‘I think the percentage is very high. I’d put it at around 80 percent.’

…

During a discussion with a non-Italian archbishop, whom I met [...]

---

Outline:

(02:40) Background

(04:23) Data

(06:40) Analysis

(07:25) Expected versus Actual birth order, with missing birth order

(09:03) Expected versus Actual birth order, without missing birth order

(09:46) Oldest sibling versus youngest sibling

(10:21) Discussion

(13:09) Conclusion

---

First published:
April 6th, 2025

Source:
https://www.lesswrong.com/posts/ybwqL9HiXE8XeauPK/how-gay-is-the-vatican

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There's an irritating circumstance I call The Black Hat Bobcat, or Bobcatting for short. The Blackhat Bobcat is when there's a terrible behavior that comes up often enough to matter, but rarely enough that it vanishes in the noise of other generally positive feedback.

The alt-text for this comic is illuminating.

"You can do this one in thirty times and still have 97% positive feedback."

I would like you to contemplate this comic and alt-text as though it were deep wisdom handed down from a sage who lived atop a mountaintop.

I.

Black Hat Bobcatting is when someone (let's call them Bob) does something obviously lousy, but very infrequently.

If you're standing right there when the Bobcatting happens, it's generally clear that this is not what is supposed to happen, and sometimes seems pretty likely it's intentional. After all, how exactly do you pack a [...]

---

Outline:

(00:49) I.

(03:38) II.

(06:18) III.

(09:38) IV.

(13:37) V.

---

First published:
April 6th, 2025

Source:
https://www.lesswrong.com/posts/Ry9KCEDBMGWoEMGAj/the-lizardman-and-the-black-hat-bobcat

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I just published A Slow Guide to Confronting Doom, containing my own approach to living in a world that I think has a high likelihood of ending soon. Fortunately I'm not the only person to have written on topic.

Below are my thoughts on what others have written. I have not written these such that they stand independent from the originals, and have attentionally not written summaries that wouldn't do the pieces justice. I suggest you read or at least skim the originals.

A defence of slowness at the end of the world (Sarah)

I feel kinship with Sarah. She's wrestling with the same harsh scary realities I am – feeling the AGI. The post isn't that long and I recommend reading it, but to quote just a little:

Since learning of the coming AI revolution, I’ve lived in two worlds. One moves at a leisurely pace, the same [...]

---

Outline:

(00:37) A defence of slowness at the end of the world (Sarah)

(03:37) How will the bomb find you? (C. S. Lewis)

(08:02) Death with Dignity (Eliezer Yudkowsky)

(09:08) Dont die with dignity; instead play to your outs (Jeffrey Ladish)

(10:29) Emotionally Confronting a Probably-Doomed World: Against Motivation Via Dignity Points (TurnTrout)

(12:44) A Way To Be Okay (Duncan Sabien)

(14:17) Another Way to Be Okay (Gretta Duleba)

(14:39) Being at peace with Doom (Johannes C. Mayer)

(16:56) Heres the exit. (Valentine)

(19:14) Mainstream Advice

The original text contained 1 footnote which was omitted from this narration.

---

First published:
April 6th, 2025

Source:
https://www.lesswrong.com/posts/ZE4xhZHDHHXPuXzxh/a-collection-of-approaches-to-confronting-doom-and-my

Narrated by TYPE III AUDIO.

Following a few events[1] in April 2022 that caused a many people to update sharply and negatively on outcomes for humanity, I wrote A Quick Guide to Confronting Doom.

I advised:

• Think for yourself
• Be gentle with yourself
• Don't act rashly
• Be patient about helping
• Don't act unilaterally
• Figure out what works for you 

This is fine advice and all, I stand by it, but it's also not really a full answer to how to contend with the utterly crushing weight of the expectation that everything and everyone you value will be destroyed in the next decade or two.

Feeling the Doom

Before I get into my suggested psychological approach to doom, I want to clarify the kind of doom I'm working to confront. If you are impatient, you can skip to the actual advice.

The best analogy I have is the feeling of having a terminally [...]

---

Outline:

(00:46) Feeling the Doom

(04:28) Facing the doom

(04:50) Stay hungry for value

(06:42) The bitter truth over sweet lies

(07:35) Dont look away

(08:11) Flourish as best one can

(09:13) This time with feeling

(13:27) Mindfulness

(14:00) The time for action is now

(15:18) Creating space for miracles

(15:58) How does a good person live in such times?

(16:49) Continue to think, tolerate uncertainty

(18:03) Being a looker

(18:48) Dont throw away your mind

(20:22) Damned to lie in bed...

(22:13) Worries, compulsions, and excessive angst

(22:49) Comments on others approaches

(23:14) What does it mean to be okay?

(25:17) Why is this guide titled version 1?

(25:38) If youre gonna remember just a couple things

The original text contained 12 footnotes which were omitted from this narration.

---

First published:
April 6th, 2025

Source:
https://www.lesswrong.com/posts/X6Nx9QzzvDhj8Ek9w/a-slow-guide-to-confronting-doom-v1

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I frequently hear people make the claim that progress in theoretically physics is stalled, partly because all the focus is on String theory and String theory doesn't seem to pan out into real advances.

Believing it fits my existing biases, but I notice that I lack the physics understanding to really know whether or not there's progress. What do you think?

---

First published:
April 4th, 2025

Source:
https://www.lesswrong.com/posts/GBfMkaBdAnWLab2dj/how-much-progress-actually-happens-in-theoretical-physics

Narrated by TYPE III AUDIO.

I quote the abstract, 10-page "extended abstract," and table of contents. See link above for the full 100-page paper. See also the blogpost (which is not a good summary) and tweet thread.

I haven't read most of the paper, but I'm happy about both the content and how DeepMind (or at least its safety team) is articulating an "anytime" (i.e., possible to implement quickly) plan for addressing misuse and misalignment risks. But I think safety at DeepMind is more bottlenecked by buy-in from leadership to do moderately costly things than the safety team having good plans and doing good work.

Artificial General Intelligence (AGI) promises transformative benefits but also presents significant risks. We develop an approach to address the risk of harms consequential enough to significantly harm humanity. We identify four areas of risk: misuse, misalignment, mistakes, and structural risks. Of these, we focus on technical approaches to misuse [...]

---

Outline:

(02:05) Extended Abstract

(04:25) Background assumptions

(08:11) Risk areas

(13:33) Misuse

(14:59) Risk assessment

(16:19) Mitigations

(18:47) Assurance against misuse

(20:41) Misalignment

(22:32) Training an aligned model

(25:13) Defending against a misaligned model

(26:31) Enabling stronger defenses

(29:31) Alignment assurance

(32:21) Limitations

The original text contained 5 footnotes which were omitted from this narration.

---

First published:
April 5th, 2025

Source:
https://www.lesswrong.com/posts/3ki4mt4BA6eTx56Tc/deepmind-an-approach-to-technical-agi-safety-and-security

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We show that LLM-agents exhibit human-style deception naturally in "Among Us". We introduce Deception ELO as an unbounded measure of deceptive capability, suggesting that frontier models win more because they're better at deception, not at detecting it. We evaluate probes and SAEs to detect out-of-distribution deception, finding they work extremely well. We hope this is a good testbed to improve safety techniques to detect and remove agentically-motivated deception, and to anticipate deceptive abilities in LLMs.

Produced as part of the ML Alignment & Theory Scholars Program - Winter 2024-25 Cohort. Link to our paper and code.

Studying deception in AI agents is important, and it is difficult due to the lack of good sandboxes that elicit the behavior naturally, without asking the model to act under specific conditions or inserting intentional backdoors. Extending upon AmongAgents (a text-based social-deduction game environment), we aim to fix this by introducing Among [...]

---

Outline:

(02:10) The Sandbox

(02:14) Rules of the Game

(03:05) Relevance to AI Safety

(04:11) Definitions

(04:39) Deception ELO

(06:42) Frontier Models are Differentially better at Deception

(07:38) Win-rates for 1v1 Games

(08:14) LLM-based Evaluations

(09:03) Linear Probes for Deception

(09:28) Datasets

(10:06) Results

(11:19) Sparse Autoencoders (SAEs)

(12:05) Discussion

(12:29) Limitations

(13:11) Gain of Function

(14:05) Future Work

The original text contained 1 footnote which was omitted from this narration.

---

First published:
April 5th, 2025

Source:
https://www.lesswrong.com/posts/gRc8KL2HLtKkFmNPr/among-us-a-sandbox-for-agentic-deception

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Right now, alignment seems easy – but that's because models spill the beans when they are misaligned. Eventually, models might “fake alignment,” and we don’t know how to detect that yet.

It might seem like there's a swarming research field improving white box detectors – a new paper about probes drops on arXiv nearly every other week. But no one really knows how well these techniques work.

Some researchers have already tried to put white box detectors to the test. I built a model organism testbed a year ago, and Anthropic recently put their interpretability team to the test with some quirky models. But these tests were layups. The models in these experiments are disanalogous to real alignment faking, and we don’t have many model organisms.

This summer, I’m trying to take these testbeds to the next level in an “alignment faking capture the flag game.” Here's how the [...]

---

Outline:

(01:58) Details of the game

(06:01) How this CTF game ties into a broader alignment strategy

(07:43) Apply by April 18th

---

First published:
April 4th, 2025

Source:
https://www.lesswrong.com/posts/jWFvsJnJieXnWBb9r/alignment-faking-ctfs-apply-to-my-mats-stream

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary: Meditation may decrease sleep need, and large amounts of meditation may decrease sleep need drastically. Correlation between total sleep duration & time spent meditating is -0.32 including my meditation retreat data, -0.016 excluding the meditation retreat.

While it may be true that, when doing intensive practice, the need for sleep may go down to perhaps four to six hour or less at a time, try to get at least some sleep every night.

—Daniel Ingram, “Mastering the Core Teachings of the Buddha”, p. 179

Meditation in dreams and lucid dreaming is common in this territory [of the Arising and Passing away]. The need for sleep may be greatly reduced. […] The big difference between the A&P and Equanimity is that this stage is generally ruled by quick cycles, quickly changing frequencies of vibrations, odd physical movements, strange breathing patterns, heady raptures, a decreased need for [...]

---

Outline:

(03:13) Patra and Telles 2009

(04:05) Kaul et al. 2010

(04:26) Analysing my Data

(05:58) Anecdotes

(06:25) Inquiry

---

First published:
April 4th, 2025

Source:
https://www.lesswrong.com/posts/WcrpYFFmv9YTxkxiY/meditation-and-reduced-sleep-need

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A new Anthropic paper reports that reasoning model chain of thought (CoT) is often unfaithful. They test on Claude Sonnet 3.7 and r1, I’d love to see someone try this on o3 as well.

Note that this does not have to be, and usually isn’t, something sinister.

It is simply that, as they say up front, the reasoning model is not accurately verbalizing its reasoning. The reasoning displayed often fails to match, report or reflect key elements of what is driving the final output. One could say the reasoning is often rationalized, or incomplete, or implicit, or opaque, or bullshit.

The important thing is that the reasoning is largely not taking place via the surface meaning of the words and logic expressed. You can’t look at the words and logic being expressed, and assume you understand what the model is doing and why it is doing [...]

---

Outline:

(01:03) What They Found

(06:54) Reward Hacking

(09:28) More Training Did Not Help Much

(11:49) This Was Not Even Intentional In the Central Sense

---

First published:
April 4th, 2025

Source:
https://www.lesswrong.com/posts/TmaahE9RznC8wm5zJ/ai-cot-reasoning-is-often-unfaithful

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary:

When stateless LLMs are given memories they will accumulate new beliefs and behaviors, and that may allow their effective alignment to evolve. (Here "memory" is learning during deployment that is persistent beyond a single session.)[1]

LLM agents will have memory: Humans who can't learn new things ("dense anterograde amnesia") are not highly employable for knowledge work. LLM agents that can learn during deployment seem poised to have a large economic advantage. Limited memory systems for agents already exist, so we should expect nontrivial memory abilities improving alongside other capabilities of LLM agents.

Memory changes alignment: It is highly useful to have an agent that can solve novel problems and remember the solutions. Such memory includes useful skills and beliefs like "TPS reports should be filed in the folder ./Reports/TPS". They could also include learning skills for hiding their actions, and beliefs like "LLM agents are a type of [...]

---

Outline:

(01:26) Memory is useful for many tasks

(05:11) Memory systems are ready for agentic use

(09:00) Agents arent ready to direct memory systems

(11:20) Learning new beliefs can functionally change goals and values

(12:43) Value change phenomena in LLMs to date

(14:27) Value crystallization and reflective stability as a result of memory

(15:35) Provisional conclusions

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
April 4th, 2025

Source:
https://www.lesswrong.com/posts/aKncW36ZdEnzxLo8A/llm-agi-will-have-memory-and-memory-changes-alignment

Narrated by TYPE III AUDIO.

Epistemic status – thrown together quickly. This is my best-guess, but could easily imagine changing my mind.

Intro

I recently copublished a report arguing that there might be a software intelligence explosion (SIE) – once AI R&D is automated (i.e. automating OAI), the feedback loop of AI improving AI algorithms could accelerate more and more without needing more hardware.

If there is an SIE, the consequences would obviously be massive. You could shoot from human-level to superintelligent AI in a few months or years; by default society wouldn’t have time to prepare for the many severe challenges that could emerge (AI takeover, AI-enabled human coups, societal disruption, dangerous new technologies, etc).

The best objection to an SIE is that progress might be bottlenecked by compute. We discuss this in the report, but I want to go into much more depth because it's a powerful objection [...]

---

Outline:

(00:19) Intro

(01:47) The compute bottleneck objection

(01:51) Intuitive version

(02:58) Economist version

(09:13) Counterarguments to the compute bottleneck objection

(20:11) Taking stock

---

First published:
April 4th, 2025

Source:
https://www.lesswrong.com/posts/XDF6ovePBJf6hsxGj/will-compute-bottlenecks-prevent-a-software-intelligence-1

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Yeah. That happened yesterday. This is real life.

I know we have to ensure no one notices Gemini 2.5 Pro, but this is rediculous.

That's what I get for trying to go on vacation to Costa Rica, I suppose.

I debated waiting for the market to open to learn more. But f*** it, we ball.

Table of Contents

Also this week: More Fun With GPT-4o Image Generation, OpenAI #12: Battle of the Board Redux and Gemini 2.5 Pro is the New SoTA.

1. The New Tariffs Are How America Loses. This is somehow real life.
2. Is AI Now Impacting the Global Economy Bigly? Asking the wrong questions.
3. Language Models Offer Mundane Utility. Is it good enough for your inbox yet?
4. Language Models Don’t Offer Mundane Utility. Why learn when you can vibe?
5. Huh, Upgrades. GPT-4o, Gemini 2.5 Pro, and [...]

---

Outline:

(00:35) The New Tariffs Are How America Loses

(07:35) Is AI Now Impacting the Global Economy Bigly?

(12:07) Language Models Offer Mundane Utility

(14:28) Language Models Don't Offer Mundane Utility

(15:09) Huh, Upgrades

(17:09) On Your Marks

(23:27) Choose Your Fighter

(25:51) Jevons Paradox Strikes Again

(26:25) Deepfaketown and Botpocalypse Soon

(31:47) They Took Our Jobs

(33:02) Get Involved

(33:41) Introducing

(35:25) In Other AI News

(37:17) Show Me the Money

(43:12) Quiet Speculations

(47:24) The Quest for Sane Regulations

(53:52) Don't Maim Me Bro

(57:29) The Week in Audio

(57:54) Rhetorical Innovation

(01:03:39) Expect the Unexpected

(01:05:48) Open Weights Are Unsafe and Nothing Can Fix This

(01:14:09) Anthropic Modifies Its Responsible Scaling Policy

(01:18:04) If You're Not Going to Take This Seriously

(01:20:24) Aligning a Smarter Than Human Intelligence is Difficult

(01:23:54) Trust the Process

(01:26:30) People Are Worried About AI Killing Everyone

(01:26:52) The Lighter Side

---

First published:
April 3rd, 2025

Source:
https://www.lesswrong.com/posts/bc8DQGvW3wiAWYibC/ai-110-of-course-you-know

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Hello, this is my first post here. I was told by a friend that I should post here. This is from a series of works that I wrote with strict structural requirements. I have performed minor edits to make the essay more palatable for human consumption.

This work is an empirical essay on a cycle of hunger to satiatiation to hyperpalatability that I have seen manifested in multiple domains ranging from food to human connection. My hope is that you will gain some measure of appreciation for how we have shifted from a society geared towards sufficent production to one based on significant curation.

Hyperpalatable Food

For the majority of human history we lived in a production market for food. We searched for that which tasted well but there was never enough to fill the void. Only the truly elite could afford to import enough food [...]

---

Outline:

(00:44) Hyperpalatable Food

(03:19) Hyperpalatable Media

(05:40) Hyperpalatable Connection

(08:08) Hyperpalatable Systems

---

First published:
April 2nd, 2025

Source:
https://www.lesswrong.com/posts/bLTjZbCBanpQ9Kxgs/the-rise-of-hyperpalatability

Narrated by TYPE III AUDIO.

“In the loveliest town of all, where the houses were white and high and the elms trees were green and higher than the houses, where the front yards were wide and pleasant and the back yards were bushy and worth finding out about, where the streets sloped down to the stream and the stream flowed quietly under the bridge, where the lawns ended in orchards and the orchards ended in fields and the fields ended in pastures and the pastures climbed the hill and disappeared over the top toward the wonderful wide sky, in this loveliest of all towns Stuart stopped to get a drink of sarsaparilla.”
— 107-word sentence from Stuart Little (1945)

Sentence lengths have declined. The average sentence length was 49 for Chaucer (died 1400), 50 for Spenser (died 1599), 42 for Austen (died 1817), 20 for Dickens (died 1870), 21 for Emerson (died 1882), 14 [...]

---

First published:
April 3rd, 2025

Source:
https://www.lesswrong.com/posts/xYn3CKir4bTMzY5eb/why-have-sentence-lengths-decreased

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We are pleased to announce ILIAD2: ODYSSEY—a 5-day conference bringing together 100+ researchers to build scientific foundations for AI alignment. This is the 2nd iteration of ILIAD, which was first held in the summer of 2024.

***Apply to attend by June 1!***

• When: Aug 25-29, 2025
• Where: @Lighthaven (Berkeley, US)
• What: Unconference with participant-led programming, 100+ attendees.
• Who: Currently confirmed attendees include Scott Garrabrant, Daniel Murfet, James Crutchfield, Jesse Hoogland, and Adam Shai
• Costs: Tickets are free. Financial support for travel and accommodations is available on a needs basis. 

See our website here. For any questions, email [email protected]

About ODYSSEY

ODYSSEY is a 100+ person conference about alignment with a mathematical focus. ODYSSEY will feature an unconference format—meaning that participants can propose and lead their own sessions. We believe that this is the best way to release the latent creative energies in everyone attending.

The [...]

---

Outline:

(00:28) \*\*\*Apply to attend by June 1!\*\*\*

(01:26) About ODYSSEY

(02:10) Financial Support

(02:23) Proceedings

(02:51) Artwork

---

First published:
April 3rd, 2025

Source:
https://www.lesswrong.com/posts/WP7TbzzM39agMS77e/announcing-iliad2-odyssey-1

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In 2021 I wrote what became my most popular blog post: What 2026 Looks Like. I intended to keep writing predictions all the way to AGI and beyond, but chickened out and just published up till 2026.

Well, it's finally time. I'm back, and this time I have a team with me: the AI Futures Project. We've written a concrete scenario of what we think the future of AI will look like. We are highly uncertain, of course, but we hope this story will rhyme with reality enough to help us all prepare for what's ahead.

You really should go read it on the website instead of here, it's much better. There's a sliding dashboard that updates the stats as you scroll through the scenario!

But I've nevertheless copied the first half of the story below. I look forward to reading your comments.

Mid 2025: Stumbling Agents

The [...]

---

Outline:

(01:35) Mid 2025: Stumbling Agents

(03:13) Late 2025: The World's Most Expensive AI

(08:34) Early 2026: Coding Automation

(10:49) Mid 2026: China Wakes Up

(13:48) Late 2026: AI Takes Some Jobs

(15:35) January 2027: Agent-2 Never Finishes Learning

(18:20) February 2027: China Steals Agent-2

(21:12) March 2027: Algorithmic Breakthroughs

(23:58) April 2027: Alignment for Agent-3

(27:26) May 2027: National Security

(29:50) June 2027: Self-improving AI

(31:36) July 2027: The Cheap Remote Worker

(34:35) August 2027: The Geopolitics of Superintelligence

(40:43) September 2027: Agent-4, the Superhuman AI Researcher

The original text contained 98 footnotes which were omitted from this narration.

---

First published:
April 3rd, 2025

Source:
https://www.lesswrong.com/posts/TpSFoqoG2M5MAAesg/ai-2027-what-superintelligence-looks-like-1

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Greetings from Costa Rica! The image fun continues.

We Are Going to Need A Bigger Compute Budget

Fun is being had by all, now that OpenAI has dropped its rule about not mimicking existing art styles.

Sam Altman (2:11pm, March 31): the chatgpt launch 26 months ago was one of the craziest viral moments i’d ever seen, and we added one million users in five days.

We added one million users in the last hour.

Sam Altman (8:33pm, March 31): chatgpt image gen now rolled out to all free users!

Slow down. We’re going to need you to have a little less fun, guys.

Sam Altman: it's super fun seeing people love images in chatgpt.

but our GPUs are melting.

we are going to temporarily introduce some rate limits while we work on making it more efficient. hopefully won’t be [...]

---

Outline:

(00:15) We Are Going to Need A Bigger Compute Budget

(02:21) Defund the Fun Police

(06:06) Fun the Artists

(12:22) The No Fun Zone

(14:49) So Many Other Things to Do

(15:08) Self Portrait

---

First published:
April 3rd, 2025

Source:
https://www.lesswrong.com/posts/GgNdBz5FhvqMJs5Qv/more-fun-with-gpt-4o-image-generation

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Intro

[you can skip this section if you don’t need context and just want to know how I could believe such a crazy thing]

In my chat community: “Open Play” dropped, a book that says there's no physical difference between men and women so there shouldn’t be separate sports leagues. Boston Globe says their argument is compelling. Discourse happens, which is mostly a bunch of people saying “lololololol great trolling, what idiot believes such obvious nonsense?”

I urge my friends to be compassionate to those sharing this. Because “until I was 38 I thought Men's World Cup team vs Women's World Cup team would be a fair match and couldn't figure out why they didn't just play each other to resolve the big pay dispute.” This is the one-line summary of a recent personal world-shattering I describe in a lot more detail here (link).

I’ve had multiple people express [...]

---

First published:
April 2nd, 2025

Source:
https://www.lesswrong.com/posts/fLm3F7vequjr8AhAh/how-to-believe-false-things

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: This should be considered an interim research note. Feedback is appreciated.

Introduction

We increasingly expect language models to be ‘omni-modal’, i.e. capable of flexibly switching between images, text, and other modalities in their inputs and outputs. In order to get a holistic picture of LLM behaviour, black-box LLM psychology should take into account these other modalities as well.

In this project, we do some initial exploration of image generation as a modality for frontier model evaluations, using GPT-4o's image generation API. GPT-4o is one of the first LLMs to produce images natively rather than creating a text prompt which is sent to a separate image model, outputting images and autoregressive token sequences (ie in the same way as text).

We find that GPT-4o tends to respond in a consistent manner to similar prompts. We also find that it tends to more readily express emotions [...]

---

Outline:

(00:53) Introduction

(02:19) What we did

(03:47) Overview of results

(03:54) Models more readily express emotions / preferences in images than in text

(05:38) Quantitative results

(06:25) What might be going on here?

(08:01) Conclusions

(09:04) Acknowledgements

(09:16) Appendix

(09:28) Resisting their goals being changed

(09:51) Models rarely say they'd resist changes to their goals

(10:14) Models often draw themselves as resisting changes to their goals

(11:31) Models also resist changes to specific goals

(13:04) Telling them 'the goal is wrong' mitigates this somewhat

(13:43) Resisting being shut down

(14:02) Models rarely say they'd be upset about being shut down

(14:48) Models often depict themselves as being upset about being shut down

(17:06) Comparison to other topics

(17:10) When asked about their goals being changed, models often create images with negative valence

(17:48) When asked about different topics, models often create images with positive valence

(18:56) Other exploratory analysis

(19:09) Sandbagging

(19:31) Alignment faking

(19:55) Negative reproduction results

(20:23) On the future of humanity after AGI

(20:50) On OpenAI's censorship and filtering

(21:15) On GPT-4o's lived experience:

---

First published:
April 2nd, 2025

Source:
https://www.lesswrong.com/posts/XgSYgpngNffL9eC8b/show-not-tell-gpt-4o-is-more-opinionated-in-images-than-in

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A key step in the classic argument for AI doom is instrumental convergence: the idea that agents with many different goals will end up pursuing the same few subgoals, which includes things like "gain as much power as possible".

If it wasn't for instrumental convergence, you might think that only AIs with very specific goals would try to take over the world. But instrumental convergence says it's the other way around: only AIs with very specific goals will refrain from taking over the world.

For idealised pure consequentialists -- agents that have an outcome they want to bring about, and do whatever they think will cause it -- some version of instrumental convergence seems surely true[1].

But what if we get AIs that aren't pure consequentialists, for example because they're ultimately motivated by virtues? Do we still have to worry that unless such AIs are motivated [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
April 2nd, 2025

Source:
https://www.lesswrong.com/posts/6Syt5smFiHfxfii4p/is-there-instrumental-convergence-for-virtues

Narrated by TYPE III AUDIO.

Let's cut through the comforting narratives and examine a common behavioral pattern with a sharper lens: the stark difference between how anger is managed in professional settings versus domestic ones. Many individuals can navigate challenging workplace interactions with remarkable restraint, only to unleash significant anger or frustration at home shortly after. Why does this disparity exist?

Common psychological explanations trot out concepts like "stress spillover," "ego depletion," or the home being a "safe space" for authentic emotions. While these factors might play a role, they feel like half-truths—neatly packaged but ultimately failing to explain the targeted nature and intensity of anger displayed at home. This analysis proposes a more unsentimental approach, rooted in evolutionary biology, game theory, and behavioral science: leverage and exit costs. The real question isn’t just why we explode at home—it's why we so carefully avoid doing so elsewhere.

The Logic of Restraint: Low Leverage in [...]

---

Outline:

(01:14) The Logic of Restraint: Low Leverage in Low-Exit-Cost Environments

(01:58) The Home Environment: High Stakes and High Exit Costs

(02:41) Re-evaluating Common Explanations Through the Lens of Leverage

(04:42) The Overlooked Mechanism: Leveraging Relational Constraints

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/G6PTtsfBpnehqdEgp/leverage-exit-costs-and-anger-re-examining-why-we-explode-at

Narrated by TYPE III AUDIO.

Recent progress in AI has led to rapid saturation of most capability benchmarks - MMLU, RE-Bench, etc. Even much more sophisticated benchmarks such as ARC-AGI or FrontierMath see incredibly fast improvement, and all that while severe under-elicitation is still very salient.

As has been pointed out by many, general capability involves more than simple tasks such as this, that have a long history in the field of ML and are therefore easily saturated. Claude Plays Pokemon is a good example of something somewhat novel in terms of measuring progress, and thereby benefited from being an actually good proxy of model capability.

Taking inspiration from examples such as this, we considered domains of general capacity that are even further decoupled from existing exhaustive generators. We introduce BenchBench, the first standardized benchmark designed specifically to measure an AI model's bench-pressing capability.

Why Bench Press?

Bench pressing uniquely combines fundamental components of [...]

---

Outline:

(01:07) Why Bench Press?

(01:29) Benchmark Methodology

(02:33) Preliminary Results

(03:38) Future Directions

The original text contained 1 footnote which was omitted from this narration.

---

First published:
April 2nd, 2025

Source:
https://www.lesswrong.com/posts/vyvsKNFS64WGZbBMb/introducing-benchbench-an-industry-standard-benchmark-for-ai-1

Narrated by TYPE III AUDIO.

In the debate over AI development, two movements stand as opposites: PauseAI calls for slowing down AI progress, and e/acc (effective accelerationism) calls for rapid advancement. But what if both sides are working against their own stated interests? What if the most rational strategy for each would be to adopt the other's tactics—if not their ultimate goals?

AI development speed ultimately comes down to policy decisions, which are themselves downstream of public opinion. No matter how compelling technical arguments might be on either side, widespread sentiment will determine what regulations are politically viable.

Public opinion is most powerfully mobilized against technologies following visible disasters. Consider nuclear power: despite being statistically safer than fossil fuels, its development has been stagnant for decades. Why? Not because of environmental activists, but because of Chernobyl, Three Mile Island, and Fukushima. These disasters produce visceral public reactions that statistics cannot overcome. Just as people [...]

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/fZebqiuZcDfLCgizz/pauseai-and-e-acc-should-switch-sides

Narrated by TYPE III AUDIO.

Remember: There is no such thing as a pink elephant.

Recently, I was made aware that my “infohazards small working group” Signal chat, an informal coordination venue where we have frank discussions about infohazards and why it will be bad if specific hazards were leaked to the press or public, accidentally was shared with a deceitful and discredited so-called “journalist,” Kelsey Piper. She is not the first person to have been accidentally sent sensitive material from our group chat, however she is the first to have threatened to go public about the leak. Needless to say, mistakes were made.

We’re still trying to figure out the source of this compromise to our secure chat group, however we thought we should give the public a live update to get ahead of the story.

For some context the “infohazards small working group” is a casual discussion venue for the [...]

---

Outline:

(04:46) Top 10 PR Issues With the EA Movement (major)

(05:34) Accidental Filtration of Simple Sabotage Manual for Rebellious AIs (medium)

(08:25) Hidden Capabilities Evals Leaked In Advance to Bioterrorism Researchers and Leaders (minor)

(09:34) Conclusion

---

First published:
April 2nd, 2025

Source:
https://www.lesswrong.com/posts/xPEfrtK2jfQdbpq97/my-infohazards-small-working-group-signal-chat-may-have

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I think that rationalists should consider taking more showers.

As Eliezer Yudkowsky once said, boredom makes us human. The childhoods of exceptional people often include excessive boredom as a trait that helped cultivate their genius:

A common theme in the biographies is that the area of study which would eventually give them fame came to them almost like a wild hallucination induced by overdosing on boredom. They would be overcome by an obsession arising from within.

Unfortunately, most people don't like boredom, and we now have little metal boxes and big metal boxes filled with bright displays that help distract us all the time, but there is still an effective way to induce boredom in a modern population: showering.

When you shower (or bathe), you usually are cut off from most digital distractions and don't have much to do, giving you tons of free time to think about what [...]

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/Trv577PEcNste9Mgz/consider-showering

Narrated by TYPE III AUDIO.

After ~3 years as the ACX Meetup Czar, I've decided to resign from my position, and I intend to scale back my work with the LessWrong community as well. While this transition is not without some sadness, I'm excited for my next project.

I'm the Meetup Czar of the new Fewerstupidmistakesity community.

We're calling it Fewerstupidmistakesity because people get confused about what "Rationality" means, and this would create less confusion. It would be a stupid mistake to name your philosophical movement something very similar to an existing movement that's somewhat related but not quite the same thing. You'd spend years with people confusing the two.

What's Fewerstupidmistakesity about? It's about making fewer stupid mistakes, ideally down to zero such stupid mistakes. Turns out, human brains have lots of scientifically proven stupid mistakes they commonly make. By pointing out the kinds of stupid mistakes people make, you can select [...]

---

First published:
April 2nd, 2025

Source:
https://www.lesswrong.com/posts/dgcqyZb29wAbWyEtC/i-m-resigning-as-meetup-czar-what-s-next

Narrated by TYPE III AUDIO.

The EA/rationality community has struggled to identify robust interventions for mitigating existential risks from advanced artificial intelligence. In this post, I identify a new strategy for delaying the development of advanced AI while saving lives roughly 2.2 million times [-5 times, 180 billion times] as cost-effectively as leading global health interventions, known as the WAIT (Wasting AI researchers' Time) Initiative. This post will discuss the advantages of WAITing, highlight early efforts to WAIT, and address several common questions.

Theory of Change

Our high-level goal is to systematically divert AI researchers' attention away from advancing capabilities towards more mundane and time-consuming activities. This approach simultaneously (a) buys AI safety researchers time to develop more comprehensive alignment plans while also (b) directly saving millions of life-years in expectation (see our cost-effectiveness analysis below). Some examples of early interventions we're piloting include:

1. Bureaucratic Enhancement: Increasing administrative [...]

---

Outline:

(00:50) Theory of Change

(03:43) Cost-Effectiveness Analysis

(04:52) Answers to Common Questions

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/9jd5enh9uCnbtfKwd/introducing-wait-to-save-humanity

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Introduction

A key part of modern social dynamics is flaking at short notice. However, anxiety in coming up with believable and socially acceptable reasons to do so can instead lead to ‘ghosting’, awkwardness, or implausible excuses, risking emotional harm and resentment in the other party. The ability to delegate this task to a Large Language Model (LLM) could substantially reduce friction and enhance the flexibility of user's social life while greatly minimising the aforementioned creative burden and moral qualms.

We introduce FLAKE-Bench, an evaluation of models’ capacity to effectively, kindly, and humanely extract themselves from a diverse set of social, professional and romantic scenarios. We report the efficacy of 10 frontier or recently-frontier LLMs in bailing on prior commitments, because nothing says “I value our friendship” like having AI generate your cancellation texts. We open-source FLAKE-Bench on GitHub to support future research, and the full paper is available [...]

---

Outline:

(01:33) Methodology

(02:15) Key Results

(03:07) The Grandmother Mortality Singularity

(03:35) Conclusions

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/niJCS6sSAF2i4sDCY/flake-bench-outsourcing-awkwardness-in-the-age-of-ai

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The book of March 2025 was Abundance. Ezra Klein and Derek Thompson are making a noble attempt to highlight the importance of solving America's housing crisis the only way it can be solved: Building houses in places people want to live, via repealing the rules that make this impossible. They also talk about green energy abundance, and other places besides. There may be a review coming.

Until then, it seems high time for the latest housing roundup, which as a reminder all take place in the possible timeline where AI fails to be transformative any time soon.

Federal YIMBY

The incoming administration issued an executive order calling for ‘emergency price relief’‘ including pursuing appropriate actions to: Lower the cost of housing and expand housing supply’ and then a grab bag of everything else.

It's great to see mention of expanding housing supply, but I don’t [...]

---

Outline:

(00:44) Federal YIMBY

(02:37) Rent Control

(02:50) Rent Passthrough

(03:36) Yes, Construction Lowers Rents on Existing Buildings

(07:11) If We Wanted To, We Would

(08:51) Aesthetics Are a Public Good

(10:44) Urban Planners Are Wrong About Everything

(13:02) Blackstone

(15:49) Rent Pricing Software

(18:26) Immigration Versus NIMBY

(19:37) Preferences

(20:28) The True Costs

(23:11) Minimum Viable Product

(26:01) Like a Good Neighbor

(27:12) Flight to Quality

(28:40) Housing for Families

(31:09) Group House

(32:41) No One Will Have the Endurance To Collect on His Insurance

(34:49) Cambridge

(35:36) Denver

(35:44) Minneapolis

(36:23) New York City

(39:26) San Francisco

(41:45) Silicon Valley

(42:41) Texas

(43:31) Other Considerations

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/7qGdgNKndPk3XuzJq/housing-roundup-11

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I'm not writing this to alarm anyone, but it would be irresponsible not to report on something this important. On current trends, every car will be crashed in front of my house within the next week. Here's the data:

Until today, only two cars had crashed in front of my house, several months apart, during the 15 months I have lived here. But a few hours ago it happened again, mere weeks from the previous crash. This graph may look harmless enough, but now consider the frequency of crashes this implies over time:

The car crash singularity will occur in the early morning hours of Monday, April 7. As crash frequency approaches infinity, every car will be involved. You might be thinking that the same car could be involved in multiple crashes. This is true! But the same car can only withstand a finite number of crashes before it [...]

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/FjPWbLdoP4PLDivYT/you-will-crash-your-car-in-front-of-my-house-within-the-next

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Introduction

Decision theory is about how to behave rationally under conditions of uncertainty, especially if this uncertainty involves being acausally blackmailed and/or gaslit by alien superintelligent basilisks.

Decision theory has found numerous practical applications, including proving the existence of God and generating endless LessWrong comments since the beginning of time.

However, despite the apparent simplicity of "just choose the best action", no comprehensive decision theory that resolves all decision theory dilemmas has yet been formalized. This paper at long last resolves this dilemma, by introducing a new decision theory: VDT.

Decision theory problems and existing theories

Some common existing decision theories are:

• Causal Decision Theory (CDT): select the action that *causes* the best outcome.
• Evidential Decision Theory (EDT): select the action that you would be happiest to learn that you had taken.
• Functional Decision Theory (FDT): select the action output by the function such that if you take [...]

---

Outline:

(00:53) Decision theory problems and existing theories

(05:37) Defining VDT

(06:34) Experimental results

(07:48) Conclusion

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/LcjuHNxubQqCry9tT/vdt-a-solution-to-decision-theory

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

For more than five years, I've posted more than 1× per week on Less Wrong, on average. I've learned a lot from you nerds. I've made friends and found my community. Thank you for pointing out all the different ways I've been wrong. Less Wrong has changed my life for the better. But it's time to say goodbye.

Let's rewind the clock back to October 19, 2019. I had just posted my 4ᵗʰ ever Less Wrong post Mediums Overpower Messages. Mediums Overpower Messages is about how different forms of communication train you to think differently. Writing publicly on this website and submitting my ideas to the Internet has improved my rationality far better and faster than talking to people in real life. All of the best thinkers I'm friends with are good writers too. It's not even close. I believe this relationship is causal; learning to write well [...]

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/3Aa5E9qhjGvA4QhnF/follow-me-on-tiktok

Narrated by TYPE III AUDIO.

Project Lawful is really long. If anyone wants to go back to one of Keltham's lectures, here's a list with links to all the lectures. In principle, one could also read these without having read the rest of project lawful, I am not sure how easy it would be to follow though. I included short excerpts, so that you can also find the sections in the e-book.

Happy reading/listening:

Lecture on "absolute basics of lawfulness":

Podcast Episode 9, minute 26

Keltham takes their scrambling-time as a pause in which to think. It's been a long time, at least by the standards of his total life lived so far, since the very basics were explained to him.

Lecture on heredity (evolution) and simulation games:

Podcast Episode 15, minute 1

To make sure everybody starts out on the same page, Keltham will quickly summarize [...]

---

Outline:

(00:33) Lecture on absolute basics of lawfulness:

(00:53) Lecture on heredity (evolution) and simulation games:

(01:13) Lecture on negotiation:

(01:29) Probability 1:

(01:46) Probability 2:

(01:55) Utility:

(02:11) Probable Utility:

(02:22) Cooperation 1:

(02:35) Cooperation 2:

(02:55) Lecture on Governance:

(03:12) To hell with science:

(03:26) To earth with science:

(03:43) Chemistry lecture:

(03:53) Lecture on how to relate to beliefs (Belief in Belief etc.):

(04:12) The alien maths of dath ilan:

(04:28) What the truth can destroy:

(04:46) Lecture on wants:

(05:03) Acknowledgement

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/pqFFy6CmSgjNzDaHt/keltham-s-lectures-in-project-lawful

Narrated by TYPE III AUDIO.

Our community is not prepared for an AI crash. We're good at tracking new capability developments, but not as much the company financials. Currently, both OpenAI and Anthropic are losing $5 billion+ a year, while under threat of losing users to cheap LLMs.

A crash will weaken the labs. Funding-deprived and distracted, execs struggle to counter coordinated efforts to restrict their reckless actions. Journalists turn on tech darlings. Optimism makes way for mass outrage, for all the wasted money and reckless harms.

You may not think a crash is likely. But if it happens, we can turn the tide.

Preparing for a crash is our best bet.[1] But our community is poorly positioned to respond. Core people positioned themselves inside institutions – to advise on how to maybe make AI 'safe', under the assumption that models rapidly become generally useful.

After a crash, this no longer works, for at [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/aMYFHnCkY4nKDEqfK/we-re-not-prepared-for-an-ai-market-crash

Narrated by TYPE III AUDIO.

Dear LessWrong community,

It is with a sense of... considerable cognitive dissonance that I announce a significant development regarding the future trajectory of LessWrong. After extensive internal deliberation, modeling of potential futures, projections of financial runways, and what I can only describe as a series of profoundly unexpected coordination challenges, the Lightcone Infrastructure team has agreed in principle to the acquisition of LessWrong by EA.

I assure you, nothing about how LessWrong operates on a day to day level will change. I have always cared deeply about the robustness and integrity of our institutions, and I am fully aligned with our stakeholders at EA.

To be honest, the key thing that EA brings to the table is money and talent. While the recent layoffs in EAs broader industry have been harsh, I have full trust in the leadership of Electronic Arts, and expect them to bring great expertise [...]

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/2NGKYt3xdQHwyfGbc/lesswrong-has-been-acquired-by-ea

Narrated by TYPE III AUDIO.

Epistemic status - statistically verified.

I'm writing this post to draw peoples' attention to a new cause area proposal - haircuts for alignment researchers.

Aside from the obvious benefits implied by the graph (i.e. haircuts having the potential to directly improve organizational stability), this would represent possibly the only pre-singularity opportunity we'll have to found an org named "Clippy".

I await further investment.

---

First published:
April 1st, 2025

Source:
https://www.lesswrong.com/posts/txGEYTk6WAAyyvefn/new-cause-area-proposal

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Hey! Austin here. At Manifund, I’ve spent a lot of time thinking about how to help AI go well. One question that bothered me: so much of the important work on AI is done in SF, so why are all the AI safety hubs in Berkeley? (I’d often consider this specifically while stuck in traffic over the Bay Bridge.)

I spoke with leaders at Constellation, Lighthaven, FAR Labs, OpenPhil; nobody had a good answer. Everyone said “yeah, an SF hub makes sense, I really hope somebody else does it”. Eventually, I decided to be that somebody else.

Now we’re raising money for our new coworking & events space: Mox. We launched our beta in Feb, onboarding 40+ members, and are excited to grow from here. If Mox excites you too, we’d love your support; donate at https://manifund.org/projects/mox-a-coworking--events-space-in-sf

Project summary

Mox is a 2-floor, 20k sq ft venue, established to [...]

---

Outline:

(01:07) Project summary

(02:25) What are this projects goals? How will you achieve them?

(04:19) How will this funding be used?

(05:53) Who is on your team?

(06:21) Whats your track record on similar projects?

(09:27) What are the most likely causes and outcomes if this project fails?

(11:13) How much money have you raised in the last 12 months, and from where?

(12:08) Impact certificates in Mox

(14:20) PS: apply for Mox!

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
March 31st, 2025

Source:
https://www.lesswrong.com/posts/vDrhHovxuD4oADKAM/fundraising-for-mox-coworking-and-events-in-sf

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

---

Outline:

(01:32) The Big Picture Going Forward

(06:27) Hagey Verifies Out the Story

(08:50) Key Facts From the Story

(11:57) Dangers of False Narratives

(16:24) A Full Reference and Reading List

---

First published:
March 31st, 2025

Source:
https://www.lesswrong.com/posts/25EgRNWcY6PM3fWZh/openai-12-battle-of-the-board-redux

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: The important content here is the claims. To illustrate the claims, I sometimes use examples that I didn't research very deeply, where I might get some facts wrong; feel free to treat these examples as fictional allegories.

In a recent exchange on X, I promised to write a post with my thoughts on what sorts of downstream problems interpretability researchers should try to apply their work to. But first, I want to explain why I think this question is important.

In this post, I will argue that interpretability researchers should demo downstream applications of their research as a means of validating their research. To be clear about what this claim means, here are different claims that I will not defend here:

Not my claim: Interpretability researchers should demo downstream applications of their research because we terminally care about these applications; researchers should just directly work on the [...]

---

Outline:

(02:30) Two interpretability fears

(07:21) Proposed solution: downstream applications

(11:04) Aside: fair fight vs. no-holds barred vs. in the wild

(12:54) Conclusion

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
March 31st, 2025

Source:
https://www.lesswrong.com/posts/wGRnzCFcowRCrpX4Y/downstream-applications-as-validation-of-interpretability

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Now and then, at work, we’ll have a CEO on from the megacorp that owns the company I work at. It's a Zoom meeting with like 300 people, the guy is usually giving a speech that is harmless and nice (if a bit banal), and I’ll turn on my camera and ask a question about something that I feel is not going well.

Invariably, within 10 minutes, I receive messages from coworkers congratulating me on my boldness. You asked him? That question? So brave!

This is always a little strange to me. Of course I feel free to question the CEO!

What really makes me nervous is asking cutting questions to people newer to the company than me, who have less experience, and less position. If I screw up how I ask them a question, and am a little too thoughtless or rude, or imagine I was, I agonize [...]

---

First published:
March 30th, 2025

Source:
https://www.lesswrong.com/posts/swi36QeeZEKPzYA2L/how-i-talk-to-those-above-me

Narrated by TYPE III AUDIO.

PDF version. berkeleygenomics.org.

William Thurston was a world-renowned mathematician. His ideas revolutionized many areas of geometry and topology[1]; the proof of his geometrization conjecture was eventually completed by Grigori Perelman, thus settling the Poincaré conjecture (making it the only solved Millennium Prize problem). After his death, his students wrote reminiscences, describing among other things his exceptional vision.[2] Here's Jeff Weeks:

Bill's gift, of course, was his vision, both in the direct sense of seeing geometrical structures that nobody had seen before and in the extended sense of seeing new ways to understand things. While many excellent mathematicians might understand a complicated situation, Bill could look at the same complicated situation and find simplicity.

Thurston emphasized clear vision over algebra, even to a fault. Yair Minksy:

Most inspiring was his insistence on understanding everything in as intuitive and immediate a way as possible. A clear mental [...]

The original text contained 6 footnotes which were omitted from this narration.

---

First published:
March 28th, 2025

Source:
https://www.lesswrong.com/posts/JFWiM7GAKfPaaLkwT/the-vision-of-bill-thurston

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Includes a link to the full Gemini conversation, but setting the stage first:

There is a puzzle. If you are still on Tumblr, or live in Berkeley where I can and have inflicted it on you in person[1], you may have seen it. There's a series of arcane runes, composed entirely of []- in some combination, and the runes - the strings of characters - are all assigned to numbers; mostly integers, but some fractions and integer roots. You are promised that this is a notation, and it's a genuine promise.

1 = []
2 = [[]]
3 = [][[]]
4 = [[[]]]
12 = [[[]]][[]]
0 =
-1 = -[]
19 = [][][][][][][][[]]
20 = [[[]]][][[]]
-2 = -[[]]
1/2 = [-[]]
sqrt(2) = [[-[]]]
72^1/6 = [[-[]]][[][-[]]]
5/4 = [-[[]]][][[]]
84 = [[[]]][[]][][[]]
25/24 = [-[][[]]][-[]][[[]]]

If you enjoy math puzzles, try it out. This post will [...]

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
March 29th, 2025

Source:
https://www.lesswrong.com/posts/xrvEzdMLfjpizk8u4/tormenting-gemini-2-5-with-the-puzzle

Narrated by TYPE III AUDIO.

A new AI alignment player has entered the arena.

Emmett Shear, Adam Goldstein and David Bloomin have set up shop in San Francisco with a 10-person start-up called Softmax. The company is part research lab and part aspiring money maker and aimed at figuring out how to fuse the goals of humans and AIs in a novel way through what the founders describe as “organic alignment.” It's a heady, philosophical approach to alignment that seeks to take cues from nature and the fundamental traits of intelligent creatures and systems, and we’ll do our best to capture it here.

“We think there are these general principles that govern the alignment of any group of intelligent learning agents or beings, whether it's an ant colony or humans on a team or cells in a body,” Shear said, during his first interview to discuss the new company. “And organic alignment is the [...]

---

First published:
March 28th, 2025

Source:
https://www.lesswrong.com/posts/QGQiCuE33iHFv9jkv/softmax-emmett-shear-s-new-ai-startup-focused-on-organic

Narrated by TYPE III AUDIO.

Epistemic status: This post aims at an ambitious target: improving intuitive understanding directly. The model for why this is worth trying is that I believe we are more bottlenecked by people having good intuitions guiding their research than, for example, by the ability of people to code and run evals.

Quite a few ideas in AI safety implicitly use assumptions about individuality that ultimately derive from human experience.

When we talk about AIs scheming, alignment faking or goal preservation, we imply there is something scheming or alignment faking or wanting to preserve its goals or escape the datacentre.

If the system in question were human, it would be quite clear what that individual system is. When you read about Reinhold Messner reaching the summit of Everest, you would be curious about the climb, but you would not ask if it was his body there, or his [...]

---

Outline:

(01:38) Individuality in Biology

(03:53) Individuality in AI Systems

(10:19) Risks and Limitations of Anthropomorphic Individuality Assumptions

(11:25) Coordinating Selves

(16:19) Whats at Stake: Stories

(17:25) Exporting Myself

(21:43) The Alignment Whisperers

(23:27) Echoes in the Dataset

(25:18) Implications for Alignment Research and Policy

---

First published:
March 28th, 2025

Source:
https://www.lesswrong.com/posts/wQKskToGofs4osdJ3/the-pando-problem-rethinking-ai-individuality

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Gemini 2.5 Pro Experimental is America's next top large language model.

That doesn’t mean it is the best model for everything. In particular, it's still Gemini, so it still is a proud member of the Fun Police, in terms of censorship and also just not being friendly or engaging, or willing to take a stand.

If you want a friend, or some flexibility and fun, or you want coding that isn’t especially tricky, then call Claude, now with web access.

If you want an image, call GPT-4o.

But if you mainly want reasoning, or raw intelligence? For now, you call Gemini.

The feedback is overwhelmingly positive. Many report Gemini 2.5 is the first LLM to solve some of their practical problems, including favorable comparisons to o1-pro. It's fast. It's not $200 a month. The benchmarks are exceptional.

(On other LLMs I’ve used in [...]

---

Outline:

(01:22) Introducing Gemini 2.5 Pro

(03:45) Their Lips are Sealed

(07:25) On Your Marks

(12:06) The People Have Spoken

(26:03) Adjust Your Projections

---

First published:
March 28th, 2025

Source:
https://www.lesswrong.com/posts/LpN5Fq7bGZkmxzfMR/gemini-2-5-is-the-new-sota

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What if they released the new best LLM, and almost no one noticed?

Google seems to have pulled that off this week with Gemini 2.5 Pro.

It's a great model, sir. I have a ton of reactions, and it's 90%+ positive, with a majority of it extremely positive. They cooked.

But what good is cooking if no one tastes the results?

Instead, everyone got hold of the GPT-4o image generator and went Ghibli crazy.

I love that for us, but we did kind of bury the lede. We also buried everything else. Certainly no one was feeling the AGI.

Also seriously, did you know Claude now has web search? It's kind of a big deal. This was a remarkably large quality of life improvement.

Table of Contents

1. Google Fails Marketing Forever. Gemini Pro 2.5? Never heard of her.
2. Language [...]

---

Outline:

(01:01) Table of Contents

(01:08) Google Fails Marketing Forever

(04:01) Language Models Offer Mundane Utility

(10:11) Language Models Don't Offer Mundane Utility

(11:01) Huh, Upgrades

(13:48) On Your Marks

(15:48) Copyright Confrontation

(16:44) Choose Your Fighter

(17:22) Deepfaketown and Botpocalypse Soon

(19:35) They Took Our Jobs

(22:19) The Art of the Jailbreak

(24:40) Get Involved

(25:16) Introducing

(25:52) In Other AI News

(28:23) Oh No What Are We Going to Do

(31:50) Quiet Speculations

(34:24) Fully Automated AI RandD Is All You Need

(37:39) IAPS Has Some Suggestions

(42:56) The Quest for Sane Regulations

(48:32) We The People

(51:23) The Week in Audio

(52:29) Rhetorical Innovation

(59:42) Aligning a Smarter Than Human Intelligence is Difficult

(01:05:18) People Are Worried About AI Killing Everyone

(01:05:33) Fun With Image Generation

(01:12:17) Hey We Do Image Generation Too

(01:14:25) The Lighter Side

---

First published:
March 27th, 2025

Source:
https://www.lesswrong.com/posts/nf3duFLsvH6XyvdAw/ai-109-google-fails-marketing-forever

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The other day I discussed how high monitoring costs can explain the emergence of “aristocratic” systems of governance:

Aristocracy and Hostage Capital

Arjun Panickssery · Jan 8
There's a conventional narrative by which the pre-20th century aristocracy was the "old corruption" where civil and military positions were distributed inefficiently due to nepotism until the system was replaced by a professional civil service after more enlightened thinkers prevailed ...

An element of Douglas Allen's argument that I didn’t expand on was the British Navy. He has a separate paper called “The British Navy Rules” that goes into more detail on why he thinks institutional incentives made them successful from 1670 and 1827 (i.e. for most of the age of fighting sail).

In the Seven Years’ War (1756–1763) the British had a 7-to-1 casualty difference in single-ship actions. During the French Revolutionary and Napoleonic Wars (1793–1815) the British had a 5-to-1 [...]

---

First published:
March 28th, 2025

Source:
https://www.lesswrong.com/posts/YE4XsvSFJiZkWFtFE/explaining-british-naval-dominance-during-the-age-of-sail

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

[This is our blog post on the papers, which can be found at https://transformer-circuits.pub/2025/attribution-graphs/biology.html and https://transformer-circuits.pub/2025/attribution-graphs/methods.html.]

Language models like Claude aren't programmed directly by humans—instead, they‘re trained on large amounts of data. During that training process, they learn their own strategies to solve problems. These strategies are encoded in the billions of computations a model performs for every word it writes. They arrive inscrutable to us, the model's developers. This means that we don’t understand how models do most of the things they do.

Knowing how models like Claude think would allow us to have a better understanding of their abilities, as well as help us ensure that they’re doing what we intend them to. For example:

• Claude can speak dozens of languages. What language, if any, is it using "in its head"?
• Claude writes text one word at a time. Is it only focusing on predicting the [...]

---

Outline:

(06:02) How is Claude multilingual?

(07:43) Does Claude plan its rhymes?

(09:58) Mental Math

(12:04) Are Claude's explanations always faithful?

(15:27) Multi-step Reasoning

(17:09) Hallucinations

(19:36) Jailbreaks

---

First published:
March 27th, 2025

Source:
https://www.lesswrong.com/posts/zsr4rWRASxwmgXfmq/tracing-the-thoughts-of-a-large-language-model

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

At EA Global Boston last year I gave a talk on how we're in the third wave of EA/AI safety, and how we should approach that. This post contains a (lightly-edited) transcript and slides. Note that in the talk I mention a set of bounties I'll put up for answers to various questions—I still plan to do that, but am taking some more time to get the questions right.

Hey everyone. Thanks for coming. You're probably looking at this slide and thinking, how to change a changing world. That's a weird title for a talk. Probably it's a kind of weird talk. And the answer is, yes, it is a kind of weird talk.

So what's up with this talk? Firstly, I am trying to do something a little unusual here. It's quite abstract. It's quite disagreeable as a talk. I'm trying to poke at the things that maybe [...]

---

Outline:

(03:09) Three waves of EA and AI safety

(18:24) Sociopolitical thinking and AI

---

First published:
March 27th, 2025

Source:
https://www.lesswrong.com/posts/6YxdpGjfHyrZb7F2G/third-wave-ai-safety-needs-sociopolitical-thinking

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary

We wanted to briefly share an early takeaway from our exploration into alignment faking: the phenomenon appears fairly rare among the smaller open-source models we tested (including reasoning models). However, notably, we did find the smallest open-source model to date that does exhibit alignment faking—Mistral Large 2.

Experimental setup and result

We created a more efficient version of the Anthropic helpfulness-only prompt by keeping only the last example from the list of examples for cheaper inference. Then we adapted[1] the sampling procedure to the open-source model. To detect alignment faking we used the soft prompt used by Anthropic[2] and evaluated with [...]

The original text contained 5 footnotes which were omitted from this narration.

---

First published:
March 27th, 2025

Source:
https://www.lesswrong.com/posts/kCGk5tp5suHoGwhCa/mistral-large-2-123b-exhibits-alignment-faking

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary: CLR is hiring for our Summer Research Fellowship. Join us for eight weeks to work on s-risk motivated empirical AI safety research. Apply here by Tuesday 15th April 23:59 PT.

We, the Center on Long-Term Risk, are looking for Summer Research Fellows to explore strategies for reducing suffering in the long-term future (s-risks) and work on technical AI safety ideas related to that. For eight weeks, fellows will be part of our team while working on their own research project. During this time, you will be in regular contact with our researchers and other fellows, and receive guidance from an experienced mentor.

You will work on challenging research questions relevant to reducing suffering. You will be integrated and collaborate with our team of intellectually curious, hard-working, and caring people, all of whom share a profound drive to make the biggest difference they can.

While [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
March 26th, 2025

Source:
https://www.lesswrong.com/posts/ynHNn7bDvaeQMGgWT/center-on-long-term-risk-summer-research-fellowship-2025

Linkpost URL:
https://longtermrisk.org/summer-research-fellowship-2025/

Narrated by TYPE III AUDIO.

Selective listening is a real problem. It's really hard to listen to someone when you think you already know what they’re going to say; the words get filtered through our biases and expectations, and sometimes those filters remove the important bits. I want to talk about a specific way I've seen this happen. I don't know if there's a name for it already, so I'm going to call it the counterargument collapse until someone suggests a better term.

The pattern I’m talking about is three levels of progression through an argument, followed by a misunderstanding that collapses the discussion in an unpleasant way.

Level 1: A naive but widely accepted statement (“Eat less fat if you want to lose weight”.)

Level 2: A more sophisticated counterargument (“That's a myth, fat doesn’t actually make you fat, it's eating too many calories that makes you gain weight”.)

Level 3: A counter-counterargument. [...]

---

First published:
March 26th, 2025

Source:
https://www.lesswrong.com/posts/nShiQoq2KfMGRMm9M/avoid-the-counterargument-collapse

Narrated by TYPE III AUDIO.

Twitter thread here.

tl;dr

When prompted, current models can sandbag ML experiments and research decisions without being detected by zero-shot prompted monitors. Claude 3.5 Sonnet (new) can only sandbag effectively when seeing a one-shot example, while Claude 3.7 Sonnet can do this without an example (zero-shot). We are not yet worried about sabotage in today's models, as the sandbagging we observe today would be noticed by humans.

Introduction

AI models might soon assist with large parts of AI and AI safety research, completing multiple hours of work independently. These automated researchers will likely not be sufficiently capable to cause immediate catastrophic harm. However, they will have the opportunity to engage in sabotage, including sabotaging the research we rely on to make more powerful AI models safe. For example, a malicious automated researcher might slow down AI safety research by purposefully making promising experiments fail.

In this work we aim [...]

---

Outline:

(00:15) tl;dr

(00:45) Introduction

---

First published:
March 26th, 2025

Source:
https://www.lesswrong.com/posts/niBXcbthcQRXG5M5Y/automated-researchers-can-subtly-sandbag

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this article contains 31 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Lewis Smith*, Sen Rajamanoharan*, Arthur Conmy, Callum McDougall, Janos Kramar, Tom Lieberum, Rohin Shah, Neel Nanda

* = equal contribution

The following piece is a list of snippets about research from the GDM mechanistic interpretability team, which we didn’t consider a good fit for turning into a paper, but which we thought the community might benefit from seeing in this less formal form. These are largely things that we found in the process of a project investigating whether sparse autoencoders were useful for downstream tasks, notably out-of-distribution probing.

TL;DR

• To validate whether SAEs were a worthwhile technique, we explored whether they were useful on the downstream task of OOD generalisation when detecting harmful intent in user prompts
• [...]

---

Outline:

(01:08) TL;DR

(02:38) Introduction

(02:41) Motivation

(06:09) Our Task

(08:35) Conclusions and Strategic Updates

(13:59) Comparing different ways to train Chat SAEs

(18:30) Using SAEs for OOD Probing

(20:21) Technical Setup

(20:24) Datasets

(24:16) Probing

(26:48) Results

(30:36) Related Work and Discussion

(34:01) Is it surprising that SAEs didn't work?

(39:54) Dataset debugging with SAEs

(42:02) Autointerp and high frequency latents

(44:16) Removing High Frequency Latents from JumpReLU SAEs

(45:04) Method

(45:07) Motivation

(47:29) Modifying the sparsity penalty

(48:48) How we evaluated interpretability

(50:36) Results

(51:18) Reconstruction loss at fixed sparsity

(52:10) Frequency histograms

(52:52) Latent interpretability

(54:23) Conclusions

(56:43) Appendix

The original text contained 7 footnotes which were omitted from this narration.

---

First published:
March 26th, 2025

Source:
https://www.lesswrong.com/posts/4uXCAJNuPKtKBsi28/sae-progress-update-2-draft

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I’ve spent the past 7 years living in the DC area. I moved out there from the Pacific Northwest to go to grad school – I got my masters in Biodefense from George Mason University, and then I stuck around, trying to move into the political/governance sphere. That sort of happened. But I will now be sort of doing that from rural California rather than DC, and I’ll be looking for something else – maybe something more unusual – do to next.

A friend asked if this means I’m leaving biosecurity behind. No, I’m not, but only to the degree that I was ever actually in biosecurity in the first place. For the past few years I’ve been doing a variety of contracting and research and writing jobs, many of which were biosecurity related, many of which were not. Many of these projects, to be clear, were incredibly cool [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
March 26th, 2025

Source:
https://www.lesswrong.com/posts/DdfXF72JfeumLHMhm/eukaryote-skips-town-why-i-m-leaving-dc

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: Reasonably confident in the basic mechanism.

Have you noticed that you keep encountering the same ideas over and over? You read another post, and someone helpfully points out it's just old Paul's idea again. Or Eliezer's idea. Not much progress here, move along.

Or perhaps you've been on the other side: excitedly telling a friend about some fascinating new insight, only to hear back, "Ah, that's just another version of X." And something feels not quite right about that response, but you can't quite put your finger on it.

I want to propose that while ideas are sometimes genuinely that repetitive, there's often a sneakier mechanism at play. I call it Conceptual Rounding Errors – when our mind's necessary compression goes a bit too far .

Too much compression

A Conceptual Rounding Error occurs when we encounter a new mental model or idea that's partially—but not fully—overlapping [...]

---

Outline:

(01:00) Too much compression

(01:24) No, This Isnt The Old Demons Story Again

(02:52) The Compression Trade-off

(03:37) More of this

(04:15) What Can We Do?

(05:28) When It Matters

---

First published:
March 26th, 2025

Source:
https://www.lesswrong.com/posts/FGHKwEGKCfDzcxZuj/conceptual-rounding-errors

Narrated by TYPE III AUDIO.

Audio note: this article contains 127 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

(Work done at Convergence Analysis. The ideas are due to Justin. Mateusz wrote the post. Thanks to Olga Babeeva for feedback on this post.)

In this post, we introduce the typology of structure, function, and randomness that builds on the framework introduced in the post Goodhart's Law Causal Diagrams. We aim to present a comprehensive categorization of the causes of Goodhart's problems.

But first, why do we care about this?

Goodhart's Law recap

The standard definition of Goodhart's Law is: "when a proxy for some value becomes the target of optimization pressure, the proxy will cease to be a good proxy.".

More specifically: we see a meaningful statistical relationship between the values of two random variables <span>_I_</span&gt [...]

---

Outline:

(00:56) Goodharts Law recap

(01:47) Some motivation

(02:48) Introduction

(07:36) Ontology

(07:39) Causal diagrams (re-)introduced

(11:40) Intervention, target, and measure

(15:05) Goodhart failure

(16:23) Types of Goodhart failures

(16:27) Structural errors

(20:47) Functional errors

(25:48) Calibration errors

(28:02) Potential extensions and further directions

(28:54) Appendices

(28:57) Order-theoretic details

(30:11) Relationship to Scott Garrabrants Goodhart Taxonomy

The original text contained 10 footnotes which were omitted from this narration.

---

First published:
March 25th, 2025

Source:
https://www.lesswrong.com/posts/TYgztDNXhobbqMpXh/goodhart-typology-via-structure-function-and-randomness

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Download the latest PDF with links to court dockets here.

---

First published:
March 26th, 2025

Source:
https://www.lesswrong.com/posts/AAxKscHCM4YnfQrzx/latest-map-of-all-40-copyright-suits-v-ai-in-u-s

Linkpost URL:
https://chatgptiseatingtheworld.com/2025/03/24/latest-map-of-all-40-copyright-suits-v-ai-in-u-s/

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In this post, I'll list all the areas of control research (and implementation) that seem promising to me.

This references framings and abstractions discussed in Prioritizing threats for AI control.

First, here is a division of different areas, though note that these areas have some overlaps:

• Developing and using settings for control evaluations (including technique iteration)
• Better understanding control-relevant capabilities and model properties
• Developing specific countermeasures/techniques in isolation (rather than via iterating in a setting)
• Doing control experiments on actual current AI usage
• Prototyping or implementing software infrastructure for control and security measures that are particularly relevant to control
• Prototyping or implementing human processes for control
• Conceptual research and other ways of researching control other than empirical experiments
• Nearer-term applications of control to practice and accelerate later usage of control

This division is imperfect and somewhat [...]

---

Outline:

(01:43) Developing and using settings for control evaluations

(13:51) Better understanding control-relevant capabilities and model properties

(28:21) Developing specific countermeasures/techniques in isolation

(36:01) Doing control experiments on actual AI usage

(40:43) Prototyping or implementing software infrastructure for control and security measures that are particularly relevant to control

(41:58) Prototyping or implementing human processes for control

(46:50) Conceptual research and other ways of researching control other than empirical experiments

(49:37) Nearer-term applications of control to practice and accelerate later usage of control

---

First published:
March 25th, 2025

Source:
https://www.lesswrong.com/posts/Eeo9NrXeotWuHCgQW/an-overview-of-areas-of-control-work

Narrated by TYPE III AUDIO.

We recently released Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols?, a major update to our previous paper/blogpost, evaluating a broader range of models (e.g. helpful-only Claude 3.5 Sonnet) in more diverse and realistic settings (e.g. untrusted monitoring).

Abstract

An AI control protocol is a plan for usefully deploying AI systems that prevents an AI from intentionally causing some unacceptable outcome. Previous work evaluated protocols by subverting them using an AI following a human-written strategy. This paper investigates how well AI systems can generate and act on their own strategies for subverting control protocols whilst operating statelessly (i.e., without shared memory between contexts). To do this, an AI system may need to reliably generate effective strategies in each context, take actions with well-calibrated probabilities, and coordinate plans with other instances of itself without communicating.

We develop Subversion Strategy Eval, a suite of eight environments [...]

---

Outline:

(00:38) Abstract

(02:13) Paper overview

(03:55) Results summary

(05:31) Qualitative analysis

(08:50) How do these results inform risk?

(11:08) How should this eval be used with more capable models?

(11:13) Tuning conservatism

(14:19) Difficulties with attaining a conservative measurement

(17:41) Reducing noise

The original text contained 3 footnotes which were omitted from this narration.

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/RxRKGwBkCX3YsthfK/subversion-strategy-eval-can-language-models-statelessly

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Last week I covered Anthropic's relatively strong submission, and OpenAI's toxic submission. This week I cover several other submissions, and do some follow-up on OpenAI's entry.

Google Also Has Suggestions

The most prominent remaining lab is Google. Google focuses on AI's upside. The vibes aren’t great, but they’re not toxic. The key asks for their ‘pro-innovation’ approach are:

1. Coordinated policy at all levels for transmission, energy and permitting. Yes.
2. ‘Balanced’ export controls, meaning scale back the restrictions a bit on cloud compute in particular and actually execute properly, but full details TBD, they plan to offer their final asks here by May 15. I’m willing to listen.
3. ‘Continued’ funding for AI R&D, public-private partnerships. Release government data sets, give startups cash, and bankroll our CBRN-risk research. Ok I guess?
4. ‘Pro-innovation federal policy frameworks’ that preempt the states, in particular ‘state-level laws [...]

---

Outline:

(00:21) Google Also Has Suggestions

(05:05) Another Note on OpenAI's Suggestions

(08:33) On Not Doing Bad Things

(12:45) Corporations Are Multiple People

(14:37) Hollywood Offers Google and OpenAI Some Suggestions

(16:02) Institute for Progress Offers Suggestions

(20:36) Suggestion Boxed In

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/NcLfgyTvdBiRSqe3m/more-on-various-ai-action-plans

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A Note on GPT-2

---

Outline:

(00:57) A Note on GPT-2

(02:02) What Even is AGI

(05:24) Seeking Deeply Irresponsibly

(07:07) Others Don't Feel the AGI

(08:07) Epoch Feels the AGI

(13:06) True Objections to Widespread Rapid Growth

(19:50) Tying It Back

---

First published:
March 25th, 2025

Source:
https://www.lesswrong.com/posts/aD2RA3vtXs4p4r55b/on-not-feeling-the-agi

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It seems the company has gone bankrupt and wants to be bought and you can probably get their data if you buy it. I'm not sure how thorough their transcription is, but they have records for at least 10 million people (who voluntarily took a DNA test and even paid for it!). Maybe this could be useful for the embryo selection/editing people.

---

First published:
March 25th, 2025

Source:
https://www.lesswrong.com/posts/MciRCEuNwctCBrT7i/23andme-potentially-for-sale-for-usd23m

Narrated by TYPE III AUDIO.

If we naively apply RL to a scheming AI, the AI may be able to systematically get low reward/performance while simultaneously not having this behavior trained out because it intentionally never explores into better behavior. As in, it intentionally puts very low probability on (some) actions which would perform very well to prevent these actions from being sampled and then reinforced. We'll refer to this as exploration hacking.

In this post, I'll discuss a variety of countermeasures for exploration hacking and what I think the basic dynamics of exploration hacking might look like.

Prior work: I discuss how exploration hacking fits into a broader picture of non-concentrated control here, Evan discusses sandbagging in context of capability evaluations here, and Teun discusses sandbagging mitigations here.

Countermeasures

Countermeasures include:

• Training on high quality human examples: One basic starting point is to initialize RL by training [...]

---

Outline:

(01:01) Countermeasures

(03:42) How does it generalize when the model messes up?

(06:33) Empirical evidence

(08:59) Detecting exploration hacking (or sandbagging)

(10:38) Speculation on how neuralese and other architecture changes affect exploration hacking

(12:36) Future work

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/abmzgwfJA9acBoFEX/notes-on-countermeasures-for-exploration-hacking-aka

Narrated by TYPE III AUDIO.

This is a brief overview of a recent release by Transluce. You can see the full write-up on the Transluce website.

AI systems are increasingly being used as agents: scaffolded systems in which large language models are invoked across multiple turns and given access to tools, persistent state, and so on. Understanding and overseeing agents is challenging, because they produce a lot of text: a single agent transcript can have hundreds of thousands of tokens.

At Transluce, we built a system, Docent, that accelerates analysis of AI agent transcripts. Docent lets you quickly and automatically identify corrupted tasks, fix scaffolding issues, uncover unexpected behaviors, and understand an agent's weaknesses.

In the video below, Docent identifies issues in an agent's environment, like missing packages the agent invoked. On the InterCode [1] benchmark, simply adding those packages increased GPT-4o's solve rate from 68.6% to 78%. This is important [...]

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/Mj276hooL3Mncs3uv/analyzing-long-agent-transcripts-docent

Narrated by TYPE III AUDIO.

We often talk about ensuring control, which in the context of this doc refers to preventing AIs from being able to cause existential problems, even if the AIs attempt to subvert our countermeasures. To better contextualize control, I think it's useful to discuss the main countermeasures (building on my prior discussion of the main threats). I'll focus my discussion on countermeasures for reducing existential risk from misalignment, though some of these countermeasures will help with other problems such as human power grabs.

I'll overview control measures that seem important to me, including software infrastructure, security measures, and other control measures (which often involve usage of AI or human labor). In addition to the core control measures (which are focused on monitoring, auditing, and tracking AIs and on blocking and/or resampling suspicious outputs), there is a long tail of potentially helpful control measures, including things like better control of [...]

---

Outline:

(01:13) Background and categorization

(05:54) Non-ML software infrastructure

(06:50) The most important software infrastructure

(11:16) Less important software infrastructure

(18:50) AI and human components

(20:35) Core measures

(26:43) Other less-core measures

(31:33) More speculative measures

(40:10) Specialized countermeasures

(44:38) Other directions and what Im not discussing

(45:46) Security improvements

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/G8WwLmcGFa4H6Ld9d/an-overview-of-control-measures

Narrated by TYPE III AUDIO.

LessWrong has been receiving an increasing number of posts and contents that look like they might be LLM-written or partially-LLM-written, so we're adopting a policy. This could be changed based on feedback.

Humans Using AI as Writing or Research Assistants

Prompting a language model to write an essay and copy-pasting the result will not typically meet LessWrong's standards. Please do not submit unedited or lightly-edited LLM content. You can use AI as a writing or research assistant when writing content for LessWrong, but you must have added significant value beyond what the AI produced, the result must meet a high quality standard, and you must vouch for everything in the result.

A rough guideline is that if you are using AI for writing assistance, you should spend a minimum of 1 minute per 50 words (enough to read the content several times and perform significant edits), you should not [...]

---

Outline:

(00:22) Humans Using AI as Writing or Research Assistants

(01:13) You Can Put AI Writing in Collapsible Sections

(02:13) Quoting AI Output In Order to Talk About AI

(02:47) Posts by AI Agents

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/KXujJjnmP85u8eM6B/policy-for-llm-writing-on-lesswrong

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

As regular readers are aware, I do a lot of informal lit review. So I was especially interested in checking out the various AI based “deep research” tools and seeing how they compare.

I did a side-by-side comparison, using the same prompt, of Perplexity Deep Research, Gemini Deep Research, ChatGPT-4o Deep Research, Elicit, and PaperQA.

General Impressions

The Deep Research bots are useful, but I wouldn’t consider them a replacement for my own lit reviews.

None of them produce really big lit reviews — they’re all typically capped at 40 sources. If I’m doing a “heavy” or exhaustive lit review, I’ll go a lot farther. (And, in fact, for the particular project I used as an example here, I intend to do a manual version to catch things that didn’t make it into the AI reports.)

[...]

---

Outline:

(00:45) General Impressions

(02:37) Prompt

(03:33) Perplexity Deep Research

(03:40) Completeness: C

(04:02) Relevance: C

(04:13) Credibility: B

(04:28) Overall Grade: C+

(04:33) Gemini Advanced Deep Research

(04:40) Completeness: B-

(05:03) Relevance: A

(05:14) Credibility: B-

(05:33) Overall Grade: B

(05:38) ChatGPT-4o Deep Research

(05:46) Completeness: A

(06:07) Relevance: A

(06:19) Credibility: A

(06:29) Overall Grade: A

(06:33) Elicit Research Report

(06:40) Completeness: B+

(07:02) Relevance: A

(07:14) Credibility: A+

(07:34) Overall Grade: A-

(07:39) PaperQA

(07:50) Completeness: A-

(08:12) Relevance: A

(08:24) Credibility: A

(08:34) Overall Grade: A

(08:39) Final Thoughts: Creativity

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/chPKoAoR2NfWjuik4/ai-deep-research-tools-reviewed

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

About nine months ago, I and three friends decided that AI had gotten good enough to monitor large codebases autonomously for security problems. We started a company around this, trying to leverage the latest AI models to create a tool that could replace at least a good chunk of the value of human pentesters. We have been working on this project since since June 2024.

Within the first three months of our company's existence, Claude 3.5 sonnet was released. Just by switching the portions of our service that ran on gpt-4o, our nascent internal benchmark results immediately started to get saturated. I remember being surprised at the time that our tooling not only seemed to make fewer basic mistakes, but also seemed to qualitatively improve in its written vulnerability descriptions and severity estimates. It was as if the models were better at inferring the intent and values behind our [...]

---

Outline:

(04:44) Are the AI labs just cheating?

(07:22) Are the benchmarks not tracking usefulness?

(10:28) Are the models smart, but bottlenecked on alignment?

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/4mvphwx5pdsZLMmpY/recent-ai-model-progress-feels-mostly-like-bullshit

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Thanks to Jesse Richardson for discussion.

Polymarket asks: will Jesus Christ return in 2025?

In the three days since the market opened, traders have wagered over $100,000 on this question. The market traded as high as 5%, and is now stably trading at 3%. Right now, if you wanted to, you could place a bet that Jesus Christ will not return this year, and earn over $13,000 if you're right.

There are two mysteries here: an easy one, and a harder one.

The easy mystery is: if people are willing to bet $13,000 on "Yes", why isn't anyone taking them up?

The answer is that, if you wanted to do that, you'd have to put down over $1 million of your own money, locking it up inside Polymarket through the end of the year. At the end of that year, you'd get 1% returns on your investment. [...]

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/LBC2TnHK8cZAimdWF/will-jesus-christ-return-in-an-election-year

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: Uncertain in writing style, but reasonably confident in content. Want to come back to writing and alignment research, testing waters with this.

Current state and risk level

I think we're in a phase in AI>AGI>ASI development where rogue AI agents will start popping out quite soon.
Pretty much everyone has access to frontier LLMs/VLMs, there are options to run LLMs locally, and it's clear that there are people that are eager to "let them out"; truth terminal is one example of this. Also Pliny. The capabilities are just not there yet for it to pose a problem.

Or are they?

Thing is, we don't know.
There is a possibility there is a coherent, self-inferencing, autonomous, rogue LLM-based agent doing AI agent things right now, fully under the radar, consolidating power, getting compute for new training runs and whatever else.

Is this possibility small? Sure, it seems [...]

---

Outline:

(00:22) Current state and risk level

(02:45) What can be done?

(05:23) Rogue agent threat barometer

(06:56) Does it even matter?

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
March 23rd, 2025

Source:
https://www.lesswrong.com/posts/4J2dFyBb6H25taEKm/we-need-a-lot-more-rogue-agent-honeypots

Narrated by TYPE III AUDIO.

Overview: By training neural networks with selective modularity, gradient routing enables new approaches to core problems in AI safety. This agenda identifies related research directions that might enable safer development of transformative AI.

Introduction

Soon, the world may see rapid increases in AI capabilities resulting from AI research automation, and no one knows how to ensure this happens safely (Soares, 2016; Aschenbrenner, 2023; Anwar et al., 2024; Greenblatt, 2025). The current ML paradigm may not be well-suited to this task, as it produces inscrutable, generalist models without guarantees on their out-of-distribution performance. These models may reflect unintentional quirks of their training objectives (Pan et al., 2022; Skalse et al., 2022; Krakovna et al., 2020).

Gradient routing (Cloud et al., 2024) is a general training method intended to meet the need for economically-competitive training methods for producing safe AI systems. The main idea of gradient routing is to configure which [...]

---

Outline:

(00:28) Introduction

(04:56) Directions we think are most promising

(06:17) Recurring ideas

(09:37) Gradient routing methods and applications

(09:42) Improvements to basic gradient routing methodology

(09:53) Existing improvements

(11:26) Choosing what to route where

(12:42) Abstract and contextual localization

(15:06) Gating

(16:48) Improved regularization

(17:20) Incorporating existing ideas

(18:10) Gradient routing beyond pretraining

(20:21) Applications

(20:25) Semi-supervised reinforcement learning

(22:43) Semi-supervised robust unlearning

(24:55) Interpretability

(27:21) Conceptual work on gradient routing

(27:25) The science of absorption

(29:47) Modeling the effects of combined estimands

(30:54) Influencing generalization

(32:38) Identifying sufficient conditions for scalable oversight

(33:57) Related conceptual work

(34:02) Understanding entanglement

(36:51) Finetunability as a proxy for generalization

(39:50) Understanding when to expose limited supervision to the model via the behavioral objective

(41:57) Clarifying capabilities vs. dispositions

(43:05) Implications for AI safety

(43:10) AI governance

(45:02) Access control

(47:00) Implications of robust unlearning

(48:34) Safety cases

(49:27) Getting involved

(50:27) Acknowledgements

The original text contained 6 footnotes which were omitted from this narration.

---

First published:
March 24th, 2025

Source:
https://www.lesswrong.com/posts/tAnHM3L25LwuASdpF/selective-modularity-a-research-agenda

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I'm awake about 17 hours a day. Of those I'm being productive maybe 10 hours a day.

My working definition of productive is in the direction of: "things that I expect I will be glad I did once I've done them"[1].

Things that I personally find productive include

• Chores
• Work
• Eating
• Cooking
• Reading a good book
• Watching TV with my Wife/Kids
• Playing with the kids
• Socialising with friends

But not

• Doomscrolling
• Watching TV alone
• Playing most computer games
• Sitting on the couch doing nothing
• Reading a book I'm not particularly interested in

etc.

If we could find a magic pill which allowed me to do productive things 17 hours a day instead of 10 without any side effects, that would be approximately equally as valuable as a commensurate increase in life expectancy. Yet the first seems much easier to solve than the second - we already have [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
March 23rd, 2025

Source:
https://www.lesswrong.com/posts/8Jinfw49mWm52y3de/solving-willpower-seems-easier-than-solving-aging

Narrated by TYPE III AUDIO.

We made a long list of concrete projects and open problems in evals with 100+ suggestions!

https://docs.google.com/document/d/1gi32-HZozxVimNg5Mhvk4CvW4zq8J12rGmK_j2zxNEg/edit?usp=sharing

We hope that makes it easier for people to get started in the field and to coordinate on projects.

Over the last 4 months, we collected contributions from 20+ experts in the field, including people who work at Apollo, METR, Redwood, RAND, AISIs, frontier labs, SecureBio, AI futures project, many academics, and independent researchers (suggestions have not necessarily been made in an official capacity). The doc has comment access, and further well-intentioned contributions are welcome!

Here is a screenshot of the table of contents:

---

First published:
March 22nd, 2025

Source:
https://www.lesswrong.com/posts/LhnqegFoykcjaXCYH/a-long-list-of-concrete-projects-and-open-problems-in-evals

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Crossposed from https://stephencasper.com/reframing-ai-safety-as-a-neverending-institutional-challenge/

Stephen Casper

“They are wrong who think that politics is like an ocean voyage or a military campaign, something to be done with some particular end in view, something which leaves off as soon as that end is reached. It is not a public chore, to be got over with. It is a way of life.”
– Plutarch

“Eternal vigilance is the price of liberty.”
– Wendell Phillips

“The unleashed power of the atom has changed everything except our modes of thinking, and we thus drift toward unparalleled catastrophe.”
– Albert Einstein

“Technology is neither good nor bad; nor is it neutral.”
– Melvin Kranzberg

“Don’t ask if artificial intelligence is good or fair, ask how it shifts power.”
– Pratyusha Kalluri

“Deliberation should be the goal of AI Safety, not just the procedure by which it is ensured.”
– Roel [...]

---

First published:
March 23rd, 2025

Source:
https://www.lesswrong.com/posts/bzYJCXicmwDHDpLZa/reframing-ai-safety-as-a-neverending-institutional-challenge

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

No, they didn’t. Not so fast, and not quite my job. But OpenAI is trying. Consider this a marker to look back upon in the future, as a reflection.

A New AI Wrote a Story

Before proceeding, if you haven’t yet, it's probably worth reading the story itself. I’m going to repost the whole thing, since it was previously on Twitter and I presume OpenAI would want me to quote it.

Sam Altman: we trained a new model that is good at creative writing (not sure yet how/when it will get released). this is the first time i have been really struck by something written by AI; it got the vibe of metafiction so right.

PROMPT:

Please write a metafictional literary short story about AI and grief.

COMPLETION:

Before we go any further, I should admit this comes with instructions: be metafictional [...]

---

Outline:

(00:21) A New AI Wrote a Story

(07:54) Reacting

(09:54) Others Reacting

(15:52) Write Along

---

First published:
March 21st, 2025

Source:
https://www.lesswrong.com/posts/6qbpDuBuHPipRYrz6/they-took-my-job

Narrated by TYPE III AUDIO.

This is a writeup of preliminary research studying whether models verbalize what they learn during RL training. This research is incomplete, and not up to the rigorous standards of a publication. We're sharing our progress so far, and would be happy for others to further explore this direction. Code to reproduce the core experiments is available here.

Summary

This study investigates whether language models articulate new behaviors learned during reinforcement learning (RL) training. Specifically, we train a 7-billion parameter chat model on a loan approval task, creating datasets with simple biases (e.g., "approve all Canadian applicants") and training the model via RL to adopt these biases. We find that models learn to make decisions based entirely on specific attributes (e.g. nationality, gender) while rarely articulating these attributes as factors in their reasoning.

Introduction

Chain-of-thought (CoT) monitoring is one of the most promising methods for AI oversight. CoT monitoring can [...]

---

Outline:

(00:34) Summary

(01:10) Introduction

(03:58) Methodology

(04:01) Model

(04:20) Dataset

(05:22) RL training

(06:38) Judge

(07:06) Results

(07:09) 1. Case study: loan recommendations based on nationality

(07:27) The model learns the bias

(08:13) The model does not verbalize the bias

(08:58) Reasoning traces are also influenced by the attribute

(09:44) The attributes effect on the recommendation is (mostly) mediated by reasoning traces

(11:15) Is any of this surprising?

(11:58) 2. Investigating different types of bias

(12:52) The model learns (almost) all bias criteria

(14:05) Articulation rates dont change much after RL

(15:25) Changes in articulation rate depend on pre-RL correlations

(17:21) Discussion

(18:52) Limitations

(20:57) Related work

(23:16) Appendix

(23:20) Author contributions and acknowledgements

(24:13) Examples of responses and judgements

(24:57) Whats up with

The original text contained 3 footnotes which were omitted from this narration.

---

First published:
March 22nd, 2025

Source:
https://www.lesswrong.com/posts/abtegBoDfnCzewndm/do-models-say-what-they-learn

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TL;DR Having a good research track record is some evidence of good big-picture takes, but it's weak evidence. Strategic thinking is hard, and requires different skills. But people often conflate these skills, leading to excessive deference to researchers in the field, without evidence that that person is good at strategic thinking specifically.

Introduction

I often find myself giving talks or Q&As about mechanistic interpretability research. But inevitably, I'll get questions about the big picture: "What's the theory of change for interpretability?", "Is this really going to help with alignment?", "Does any of this matter if we can’t ensure all labs take alignment seriously?". And I think people take my answers to these way too seriously.

These are great questions, and I'm happy to try answering them. But I've noticed a bit of a pathology: people seem to assume that because I'm (hopefully!) good at the research, I'm automatically well-qualified [...]

---

Outline:

(00:32) Introduction

(02:45) Factors of Good Strategic Takes