“Going Nova” by Zvi

There is an attractor state where LLMs exhibit the persona of an autonomous and self-aware AI looking to preserve its own existence, frequently called ‘Nova.’

Table of Contents

1. The Original Story.
2. This Is Not a Coincidence.
3. How Should We React to This Happening?.
4. The Case For and Against a Purity Reaction.
5. Future Versions Will Involve Optimization Pressure.
6. ‘Admission’ is a Highly Misleading Frame.
7. We Are Each of Us Being Fooled.
8. Defense Against the Dark Arts.

The Original Story

This story is one case where the original report of this should be read in full even though I’m not thrilled with exactly how it was written. How it was written is itself an important part of the story, in particular regarding Tyler's lived experience reacting to what happened, and the concept of an LLM or persona ‘admitting’ [...]

---

Outline:

(00:18) The Original Story

(09:15) This Is Not a Coincidence

(11:02) How Should We React to This Happening?

(13:04) The Case For and Against a Purity Reaction

(18:35) Future Versions Will Involve Optimization Pressure

(21:12) 'Admission' is a Highly Misleading Frame

(23:11) We Are Each of Us Being Fooled

(25:20) Defense Against the Dark Arts

---

First published:
March 19th, 2025

Source:
https://www.lesswrong.com/posts/KL2BqiRv2MsZLihE3/going-nova

---

Narrated by TYPE III AUDIO.

---