"Yeah, I think that's just the utilityfunction, probably, in the end. We're not coherent and we don't know what our values are, we can learn facts and arguments that change what it is we think we want and even what it is that we actually want, but - all of that is ultimately inside a framework that you're born with, that can never become known to you - and there was a very long and very hard-fought argument inside Civilization, between the 'negative utilitarians' and everyone else, before both sides came to accept that the other side wasn't making a mistake. The negative utilitarians didn't want intelligent life colonizing the universe, if that meant running a risk that people in any significant numbers would ever feel any amount of pain and unhappiness, even a small amount. That was, so far as anyone could tell, just their utilityfunction and their framework they'd been born with. If it'd been a question of the very smart people coming up with the right argument to talk them out of it - everything would have been simpler and much much less creepy."
"This is all before I was born, by the way, but an endlessly famous part of history because it's, like, one of the greatest moral stress-tests that Civilization was ever subjected to, and it generated so much drama, and the way it ended sure did not help."