Motivated Reasoning As Mis-applied Reinforcement Learning

https://astralcodexten.substack.com/p/motivated-reasoning-as-mis-applied

Here’s something else I got from the first Yudkowsky-Ngo dialogue:

Suppose you go to Lion Country and get mauled by lions. You want the part of your brain that generates plans like “go to Lion Country” to get downgraded in your decision-making algorithms. This is basic reinforcement learning: plan → lower-than-expected hedonic state → do plan less. Plan → higher-than-expected hedonic state → do plan more. Lots of brain modules have this basic architecture; if you have a foot injury and walking normally causes pain, that will downweight some basic areas of the motor cortex and make you start walking funny (potentially without conscious awareness).

But suppose you see a lion, and your visual cortex processes the sensory signals and decides “Yup, that’s a lion”. Then you have to freak out and run away, and it ruins your whole day. That’s a lower-than-expected hedonic state! If your visual cortex was fundamentally a reinforcement learner, it would learn not to recognize lions (and then the lion would eat you). So the visual cortex (and presumably lots of other sensory regions) doesn’t do hedonic reinforcement learning in the same way.

So there are two types of brain region: basically behavioral (which hedonic reinforcement learning makes better), and basically epistemic (which hedonic reinforcement learning would make worse, so they don’t do it).