The explanation to tiny Keltham resumed the next day with the workings of Governance.

Conceptually and to first-order, the ideal that Civilization is approximating is a giant macroagent composed of everybody in the world, taking coordinated macroactions to end up on the multi-agent-optimal frontier, at a point along that frontier reflecting a fair division of the gains from that coordinated macroaction -

Well, to be clear, the dath ilani would shut it all down if actual coordination levels started to get anywhere near that.  Civilization has spoken - with nearly one voice, in fact - that it does not want to turn into a hivemind.  This is why 'dath ilan' deliberately doesn't have Baseline's agency-marker on it, like the name of a person; dath ilan is not allowed to become a person.  It is high on the list of Things Dath Ilan Is Not Allowed To Do.  There was a poll once - put forth either by wacky trolls or sincere negative utilitarians - over how many people would, if they were voting on it directly, vote to put the agency-marker back into 'Dath Ilan'; 98% of respondents said no.

Dath ilan has decided to definitely not turn into a hivemind.  If it ever starts to get even close to that, everyone in Civilization will decide in nearly unanimous accord that they would rather do something else which is not that, and end up not there.  Conformity levels are bad enough already, according to their democracy's collective vote on the desired levels of that!  And predicted to get slightly worse over the next 10 years, according to the prediction markets that aggregate all of Civilization's knowledge into a single opinion that represents what Civilization as a whole can be said to know about any future observable, which few sane people would dare to question too much even in the privacy of their own thoughts!

But for moral purposes, for purposes of understanding what 'Civilization' represents as a decision individuals make to coordinate among themselves, it represents moving partway towards aggregating all coordinating parties in dath ilan into one macroagent that muxes their utility functions together, at a weighting that ends up giving every subagent a fair share of the gains according to their individual utility functions.

The explanation to tiny Keltham resumed the next day with the workings of Governance.

Conceptually and to first-order, the ideal that Civilization is approximating is a giant macroagent composed of everybody in the world, taking coordinated macroactions to end up on the multi-agent-optimal frontier, at a point along that frontier reflecting a fair division of the gains from that coordinated macroaction -

Well, to be clear, the dath ilani would shut it all down if actual coordination levels started to get anywhere near that.  Civilization has spoken - with nearly one voice, in fact - that it does not want to turn into a hivemind.  This is why 'dath ilan' deliberately doesn't have Baseline's agency-marker on it, like the name of a person; dath ilan is not allowed to become a person.  It is high on the list of Things Dath Ilan Is Not Allowed To Do.  There was a poll once - put forth either by wacky trolls or sincere negative utilitarians - over how many people would, if they were voting on it directly, vote to put the agency-marker back into 'Dath Ilan'; 98% of respondents said no.

Dath ilan has decided to definitely not turn into a hivemind.  If it ever starts to get even close to that, everyone in Civilization will decide in nearly unanimous accord that they would rather do something else which is not that, and end up not there.  Conformity levels are bad enough already, according to their democracy's collective vote on the desired levels of that!  And predicted to get slightly worse over the next 10 years, according to the prediction markets that aggregate all of Civilization's knowledge into a single opinion that represents what Civilization as a whole can be said to know about any future observable, which few sane people would dare to question too much even in the privacy of their own thoughts!

But for moral purposes, for purposes of understanding what 'Civilization' represents as a decision individuals make to coordinate among themselves, it represents moving partway towards aggregating all coordinating parties in dath ilan into one macroagent that weighted-sums their utility functions together, at a weighting that ends up giving every subagent a fair share of the gains according to their individual utility functions.