“Dream, Truth, & Good” by abramdemski

One way in which I think current AI models are sloppy is that LLMs are trained in a way that messily merges the following "layers":

• The "dream machine" layer: LLMs are pre-trained on lots of slop from the internet, which creates an excellent "prior".
• The "truth machine": LLMs are trained to "reduce hallucinations" in a variety of ways, including RLHF and the more recent reasoning RL.
• The "good machine": The same RLHF and reasoning RL training also aims to train good outputs (eg helpful, honest, harmless). 

I've quoted Andrej Karpathy before, but I'll do it again:

I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.
[...]
I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest [...]

---

Outline:

(02:13) A Modest Proposal

(02:17) Dream Machine Layer

(04:17) Truth Machine Layer

(06:13) Good Machine Layer

The original text contained 1 footnote which was omitted from this narration.

---

First published:
February 24th, 2025

Source:
https://www.lesswrong.com/posts/DPjvL62kskHpp2SZg/dream-truth-and-good

---

Narrated by TYPE III AUDIO.