Of course the real Esta is more complicated than this in all sorts of ways.

Esta likes money, for example.  This is not because Golarion hominids have evolved built-in circuitry to recognize and pursue money, the way that they have snake-recognizer circuitry or skull-fearing circuitry or that sweet tastes are reinforcing.  Esta does not even like money for some reason as simple and recognizable as money-acquisition being reinforced by orgasms and money-loss being accompanied by electrical shocks.  Esta likes money after his brain repeatedly encountered tasks in which "did you acquire / lose money" was treated as a success / failure condition without nonlocal uses of money being important enough in their details to locally think about them.  Subsequently his brain learned a habit within its planning, to treat money as if it were a terminal goal; and the step of acquiring money began to feel like success and be pleasurable as a success.  It is not that acquiring money became associated with sex or food, but that it became associated with the nice successful feeling of plans to acquire money.

So Esta does means-end planning around money, and even on an instinctive sort of level where the anticipation of losing money feels bad in nearly the same way that the anticipation of an electrical shock feels bad.  But it's not as unconditional as an electric shock, because if Esta were thrown onto another planet where money was meaningless and he had no prospect of ever returning to Golarion, the learned pleasure of finding a large hunk of gold would be diminished by the thought of having nowhere to spend that gold.

This matters because Esta's buggy state is not as simple as 'his entrained and memorized patterns of thought no longer harmonize with his innate steering toward well-anticipated pleasure and away from well-anticipated pain as were both hardwired in'.  Esta has learned habits of planning and habits of pleasure about outcomes that weren't built into him.

The prospect of embarking on a chain of thought that will end in heresy, now feels bad to Esta in the same mentally-unconditional way as the anticipation of losing money.  That isn't just about the Church having associated heresy with pain, but having successfully associated it inside him with failure at the task of correct thinking.

Of course the real Esta is more complicated than this in all sorts of ways.

Esta likes money, for example.  This is not because Golarion hominids have evolved built-in circuitry to recognize and pursue money, the way that they have snake-recognizer circuitry or skull-fearing circuitry or that sweet tastes are reinforcing.  Esta does not even like money for some reason as simple and recognizable as money-acquisition being reinforced by orgasms and money-loss being accompanied by electrical shocks.  Esta likes money after his brain repeatedly encountered tasks in which "did you acquire / lose money" was treated as a success / failure condition without nonlocal uses of money being important enough in their details to locally think about them.  Subsequently his brain learned a habit within its planning, to treat money as if it were a terminal goal; and the step of acquiring money began to feel like success and be pleasurable as a success.  It is not that acquiring money became associated with sex or food, but that it became associated with the nice successful feeling of plans to acquire money.

So Esta does means-end planning around money, and even on an instinctive sort of level where the anticipation of losing money feels bad in nearly the same way that the anticipation of an electrical shock feels bad.  But it's not as unconditional as an electric shock, because if Esta were thrown onto another planet where money was meaningless and he had no prospect of ever returning to Golarion, the learned pleasure of finding a large hunk of gold would be diminished by the thought of having nowhere to spend that gold.

This matters because Esta's buggy state is not as simple as 'his entrained and memorized patterns of thought no longer harmonize with his innate steering toward well-anticipated pleasure and away from well-anticipated pain as were both hardwired in'.  Esta has learned habits of planning and habits of pleasure about outcomes that weren't built into him.

The prospect of embarking on a chain of thought that will end in heresy, now feels bad to Esta in the same mentally-unconditional way as the anticipation of losing money.  That isn't just about the Church having associated heresy with pain, but having successfully associated it inside him with failure at the task of correct thinking.

Which is to say: where something falls on the spectrum between memorization and general planning, is not a matter of what is built-in versus learned, either in principle or inside the practical Esta.

Of course the real Esta is more complicated than this in all sorts of ways.

Esta likes money, for example.  This is not because Golarion hominids have evolved built-in circuitry to recognize and pursue money, the way that they have snake-recognizer circuitry or skull-fearing circuitry or that sweet tastes are reinforcing.  Esta does not even like money for some reason as simple and recognizable as money-acquisition being reinforced by orgasms and money-loss being accompanied by electrical shocks.  Esta likes money after his brain repeatedly encountered tasks in which "did you acquire / lose money" was treated as a success / failure condition without nonlocal uses of money being important enough in their details to locally think about them.  Subsequently his brain learned a habit within its planning, to treat money as if it were a terminal goal; and the step of acquiring money began to feel like success and be pleasurable as a success.  It is not that acquiring money became associated with sex or food, but that it became associated with the nice successful feeling of plans to acquire money.

So Esta does means-end planning around money, and even on an instinctive sort of level where the anticipation of losing money feels bad in nearly the same way that the anticipation of an electrical shock feels bad.  But it's not as unconditional as an electric shock, because if Esta were thrown onto another planet where money was meaningless and he had no prospect of ever returning to Golarion, the learned pleasure of finding a large hunk of gold would be diminished by the thought of having nowhere to spend that gold.

This matters because Esta's buggy state is not as simple as 'his entrained and memorized patterns of thought no longer harmonize with his innate steering toward well-anticipated pleasure and away from well-anticipated pain as were both hardwired in'.  Esta now has learned habits of planning, and habits of pleasure about outcomes, that weren't built into his genes.

The prospect of embarking on a chain of thought that will end in heresy, now feels bad to Esta in the same mentally-unconditional way as the anticipation of losing money.  That isn't just about the Church having associated heresy with pain, but having successfully associated it inside him with failure at the task of correct thinking.

Which is to say: where something falls on the spectrum between memorization and general planning, is not a matter of what is built-in versus learned, either in principle or inside the practical Esta.  It is just about the depth of its generality; the number of cases it covers and how much wider those cases are than the training dataset.  A cognitive element can be a little planny and narrowly wide; it can project backwards in time along only a narrow subpart of a world-model.

Of course the real Esta is more complicated than this in all sorts of ways.

Esta likes money, for example.  This is not because Golarion hominids have evolved built-in circuitry to recognize and pursue money, the way that they have snake-recognizer circuitry or skull-fearing circuitry or that sweet tastes are reinforcing.  Esta does not even like money for some reason as simple and recognizable as money-acquisition being reinforced by orgasms and money-loss being accompanied by electrical shocks.  Esta likes money after his brain repeatedly encountered tasks in which "did you acquire / lose money" was treated as a success / failure condition without nonlocal uses of money being important enough in their details to locally think about them.  Subsequently his brain learned a habit within its planning, to treat money as if it were a terminal goal; and the step of acquiring money began to feel like success and be pleasurable as a success.  It is not that acquiring money became associated with sex or food, but that it became associated with the nice successful feeling of plans to acquire money.

So Esta does means-end planning around money, and even on an instinctive sort of level where the anticipation of losing money feels bad in nearly the same way that the anticipation of an electrical shock feels bad.  But it's not as unconditional as an electric shock, because if Esta were thrown onto another planet where money was meaningless and he had no prospect of ever returning to Golarion, the learned pleasure of finding a large hunk of gold would be diminished by the thought of having nowhere to spend that gold.

This matters because Esta's buggy state is not as simple as 'his entrained and memorized patterns of thought no longer harmonize with his innate steering toward well-anticipated pleasure and away from well-anticipated pain as were both hardwired in'.  Esta now has learned habits of planning, and habits of pleasure about outcomes, that weren't built into his genes.

The prospect of embarking on a chain of thought that will end in heresy, now feels bad to Esta in the same mentally-unconditional way as the anticipation of losing money.  That isn't just about the Church having associated heresy with pain, but having successfully associated it inside him with failure at the task of correct thinking.

Which is to say: where something falls on the spectrum between memorization and general planning, is not a matter of what is built-in versus learned, either in principle or inside the practical Esta.  It is just about the depth of its generality: the number of cases it covers, and how much wider those cases are than the training dataset.  A cognitive element can be a little planny and narrowly wide; it can project backwards in time along only a narrow subpart of a world-model.

Of course the real Esta is more complicated than this in all sorts of ways.

Esta likes money, for example.  This is not because Golarion hominids have evolved built-in circuitry to recognize and pursue money, the way that they have snake-recognizer circuitry or skull-fearing circuitry or that sweet tastes are reinforcing.  Esta does not even like money for some reason as simple and recognizable as money-acquisition being reinforced by orgasms and money-loss being accompanied by electrical shocks.  Esta likes money after his brain repeatedly encountered tasks in which "did you acquire / lose money" was treated as a success / failure condition without nonlocal uses of money being important enough in their details to locally think about them.  Subsequently his brain learned a habit within its planning, to treat money as if it were a terminal goal; and the step of acquiring money began to feel like success and be pleasurable as a success.  It is not that acquiring money became associated with sex or food, but that it became associated with the nice successful feeling of plans to acquire money.

So Esta does means-end planning around money, and even on an instinctive sort of level where the anticipation of losing money feels bad in nearly the same way that the anticipation of an electrical shock feels bad.  But it's not as unconditional as an electric shock, because if Esta were thrown onto another planet where money was meaningless and he had no prospect of ever returning to Golarion, the learned pleasure of finding a large hunk of gold would be diminished by the thought of having nowhere to spend that gold.

This matters because Esta's buggy state is not as simple as 'his entrained and memorized patterns of thought no longer harmonize with his innate steering toward anticipated pleasure and away from anticipated pain as were both hardwired in'.  Esta now has learned habits of planning, and habits of pleasure about outcomes, that weren't built into his genes.

The prospect of embarking on a chain of thought that will end in heresy, now feels bad to Esta in the same mentally-unconditional way as the anticipation of losing money.  That isn't just about the Church having associated heresy with pain, but having successfully associated it inside him with failure at the task of correct thinking.

Which is to say: where something falls on the spectrum between memorization and general planning, is not a matter of what is built-in versus learned, either in principle or inside the practical Esta.  It is just about the depth of its generality: the number of cases it covers, and how much wider those cases are than the training dataset.  A cognitive element can be a little planny and narrowly wide; it can project backwards in time along only a narrow subpart of a world-model.