"I know how to reason about large numbers, which is only a matter of knowing how to carry out a whole chain of reasoning using numbers on every step.  But in this case it's not about the large-number part of it.  You value your experience of magic.  You value there being more Carissalike things that are real.  If your value on the second thing scales linearly and is currently strong enough for you to notice it at all, you'd sacrifice your future experience of magic to become two trillion times more real."

"Hell doesn't need to do the part where it reasons about large numbers.  Hell needs to know whether your currently stated preference for there to be more reality in Carissalike things, is a valid part of your idealized utilityfunction, and one that grows as an independent component without limit until it trumps the part of your idealized utilityfunction that wants to go on experiencing being able to do magic."

"And - I would personally say - giving up your sense of joy in doing magic, because you had a sense that you ought to be obligated to become two trillion times more real even if you couldn't feel that at all from the inside, even if what became two trillion times as real was the Carissa who lost her magic - that does feel to me like it has - something in common with the thinking of the people who thought they had go on existing to destroy the universe."

"There's - something of Goodness - in caring too much about what's real and not how things feel to you."

"I know how to reason about large numbers, which is only a matter of knowing how to carry out a whole chain of reasoning using numbers on every step.  But in this case it's not about the large-number part of it.  You value your experience of magic.  You value there being more Carissalike things that are real.  If your value on the second thing scales linearly and is currently strong enough for you to notice it at all, you'd sacrifice your future experience of magic to become two trillion times more real."

"Hell doesn't need to do the part where it reasons about large numbers.  Hell needs to know whether your currently stated preference for there to be more reality in Carissalike things, is a valid part of your idealized utilityfunction, and one that grows as an independent component without limit until it trumps the part of your idealized utilityfunction that wants to go on experiencing being able to do magic."

"And - I would personally say - giving up your sense of joy in doing magic, because you had a sense that you ought to be obligated to become two trillion times more real even if you couldn't feel that at all from the inside, even if what became two trillion times as real was the Carissa who lost her magic - that does feel to me like it has - something in common with the thinking of the people who thought they had go on existing to destroy the universe."

"There's - something of Goodness - in caring too much about which things become more real, and not how things feel to you."

"I know how to reason about large numbers, which is only a matter of knowing how to carry out a whole chain of reasoning using numbers on every step.  But in this case it's not about the large-number part of it.  You value your experience of magic.  You value there being more Carissalike things that are real.  If your value on the second thing scales linearly and is currently strong enough for you to notice it at all, you'd sacrifice your future experience of magic to become two trillion times more real."

"Hell doesn't need to do the part where it reasons about large numbers.  Hell needs to know whether your currently stated preference for there to be more reality in Carissalike things, is a valid part of your idealized utilityfunction, and one that grows as an independent component without limit until it trumps the part of your idealized utilityfunction that wants to go on experiencing being able to do magic."

"And - I would personally say - giving up your sense of joy in doing magic, because you had a sense that you ought to be obligated to become two trillion times more real even if you couldn't feel that at all from the inside, even if what became two trillion times as real was the Carissa who lost her magic - that does feel to me like it has - something in common with the thinking of the people who thought they had go on existing to destroy the universe."

"There's - something of Goodness - in caring too much about which things become more real, and having that drown out how things feel to you."

"I know how to reason about large numbers, which is only a matter of knowing how to carry out a whole chain of reasoning using numbers on every step.  But in this case it's not about the large-number part of it.  You value your experience of magic.  You value there being more Carissalike things that are real.  If your value on the second thing scales linearly and is currently strong enough for you to notice it at all, you'd sacrifice your future experience of magic to become two trillion times more real."

"Hell doesn't need to do the part where it reasons about large numbers.  Hell needs to know whether your currently stated preference for there to be more reality in Carissalike things, is a valid part of your idealized utilityfunction, and one that grows as an independent component without limit until it trumps the part of your idealized utilityfunction that wants to go on experiencing being able to do magic."

"And - I would personally say - giving up your sense of joy in doing magic, because you had a sense that you ought to be obligated to become two trillion times more real even if you couldn't feel that at all from the inside, even if what became two trillion times as real was the sad Carissa who lost her magic - that does feel to me like - it has something in common with the thinking of the people who thought they had go on existing in the present in order to destroy the future."

"There's - something of Goodness - in caring too much about which things become more real, and having that drown out how things feel to you."

"I know how to reason about large numbers, which is only a matter of knowing how to carry out a whole chain of reasoning using numbers on every step.  But in this case it's not about the large-number part of it.  You value your experience of magic.  You value there being more Carissalike things that are real.  If your value on the second thing scales linearly and is currently strong enough for you to notice it at all, you'd sacrifice your future experience of magic to become two trillion times more real."

"Hell doesn't need to do the part where it reasons about large numbers.  Hell needs to know whether your currently stated preference for there to be more reality in Carissalike things, is a valid part of your idealized utilityfunction, and one that grows as an independent component without limit until it trumps the part of your idealized utilityfunction that wants to go on experiencing being able to do magic."

"And - I would personally say - giving up your sense of joy in doing magic, because you had a sense that you ought to be obligated to become two trillion times more real even if you couldn't feel that at all from the inside, even if what became two trillion times as real was the sad Carissa who lost her magic - that does feel to me like - it has something in common with the thinking of the people who thought they had go on existing in the present in order to destroy the future."

"There's - something of Goodness - in caring too much about which things are more real, and having that drown out how things feel to you."

"I know how to reason about large numbers, which is only a matter of knowing how to carry out a whole chain of reasoning using numbers on every step.  But in this case it's not about the large-number part of it.  You value your experience of magic.  You value there being more Carissalike things that are real.  If your value on the second thing scales linearly and is currently strong enough for you to notice it at all, you'd sacrifice your future experience of magic to become two trillion times more real."

"Hell doesn't need to do the part where it reasons about large numbers.  Hell needs to know whether your currently stated preference for there to be more reality in Carissalike things, is a valid part of your idealized utilityfunction, and one that grows as an independent component without limit until it trumps the part of your idealized utilityfunction that wants to go on experiencing being able to do magic."

"And - I would personally say - giving up your sense of joy in doing magic, because you had a sense that you ought to be obligated to become two trillion times more real even if you couldn't feel that at all from the inside, even if what became two trillion times as real was the sad Carissa who lost her magic - that does feel to me like - it has something in common with the thinking of the people who thought they had go on existing in the present in order to destroy the future."

"There's - something of Goodness - in caring too much about what's real, and having that drown out how things feel to you."