"'Preregistration', you describe the experiment before you perform it and open a prediction market on the results before they're in.  That forces Civilization to make its own prediction in advance.  It lets anybody object beforehand instead of afterwards if they have an issue with your experimental procedure, and makes them look less persuasive if they wait until you get a result they don't like, to claim something was wrong with your methodology.  It guards against a scenario where people think that boring or expected results are 'failures' and don't bother to report on those, until somebody by random luck gets a misleading exciting result and reports on that."

"But, uh, I'm realizing as I say it, that none of this stuff about prediction markets is really the first thing we need to think about on a mini-Science! project the size of Project Lawful.  We're small enough that we can just know about all the experiments everyone is doing, and say all our individual probabilities on them out loud in advance."

"I'm sort of flailing here, going through things in random order, as you can probably tell.  In dath ilan we all sort of grow up knowing how Science! is supposed to work, if only vaguely to start with, so at no point does anybody try to teach you the whole thing all at once.  There's just little pieces here and there where you learn in more detail how some bit of Science! works."

"'Preregistration', you describe the experiment before you perform it and open a prediction market on the results before they're in.  That forces Civilization to make its own prediction in advance.  It lets anybody object beforehand instead of afterwards, if they have an issue with your experimental procedure.  It makes them look less persuasive if they wait until you get a result they don't like, to claim something was wrong with your methodology.  On a Civilizational level, we were told that guards against a scenario where people think that boring or expected results are 'failures' and don't bother to report on those, until somebody by random luck gets a misleading exciting result and reports on that."

"There's a whole dramatic debate about whether or not we ought to pay any attention to results that aren't preregistered if somebody reports on them anyways, even though that basically never happens in real life, because of how, if you say we should throw the results like that away, it means we're virtuously refusing to update on what is in fact evidence, but conversely, if we pay attention to results like that, we're creating potentially bad systemic incentives.  To the extent it ever came up in real life, I figure people would just toss the original result and pay a 'replicator' to do it over again, if it was anything important."

"But, uh, I'm realizing as I say it, that none of this stuff about prediction markets is really the first thing we need to think about on a mini-Science! project the size of Project Lawful.  We're small enough that we can just know about all the experiments everyone is doing, and say all our individual probabilities on them out loud in advance.  We don't have professional replicators to appeal to, all we can do is basically run our own experiments twice if they turn up anything surprising."

"I'm sort of flailing here, going through things in random order, as you can probably tell.  In dath ilan we all sort of grow up knowing how Science! is supposed to work, if only vaguely to start with, so at no point does anybody try to teach you the whole thing all at once.  There's just little pieces here and there where you learn in more detail how some bit of Science! works."

"'Preregistration', you describe the experiment before you perform it and open a prediction market on the results before they're in.  That forces Civilization to make its own prediction in advance.  It lets anybody object beforehand instead of afterwards, if they have an issue with your experimental procedure.  It makes them look less persuasive if they wait until you get a result they don't like, to claim something was wrong with your methodology.  On a Civilizational level, we were told that guards against a scenario where people think that boring or expected results are 'failures' and don't bother to report on those, until somebody by random luck gets a misleading exciting result and reports on that."

"There's a whole dramatic debate about whether or not we ought to pay any attention to results that aren't preregistered if somebody reports on them anyways, even though that basically never happens in real life, because of how, if you say we should throw the results like that away, it means we're virtuously refusing to update on what is in fact evidence, but conversely, if we pay attention to results like that, we're creating potentially bad systemic incentives.  To the extent it ever came up in real life, I figure people would just toss the original result and pay a 'replicator' to do it over again, if it was anything important."

"But, uh, I'm realizing as I say it, that none of this stuff about prediction markets is really the first thing we need to think about on a mini-Science! project the size of Project Lawful.  We're small enough that we can just know about all the experiments everyone is doing, and say all our individual probabilities on them out loud in advance.  We don't have professional replicators to appeal to, all we can do basically is run any important experiments twice with a different researcher in charge."

"I'm sort of flailing here, going through things in random order, as you can probably tell.  In dath ilan we all sort of grow up knowing how Science! is supposed to work, if only vaguely to start with, so at no point does anybody try to teach you the whole thing all at once.  There's just little pieces here and there where you learn in more detail how some bit of Science! works."

"'Preregistration', you describe the experiment before you perform it and open a prediction market on the results before they're in.  That forces Civilization to make its own prediction in advance.  It lets anybody object beforehand instead of afterwards, if they have an issue with your experimental procedure.  It makes them look less persuasive if they wait until you get a result they don't like, to claim something was wrong with your methodology.  On a Civilizational level, we were told that guards against a scenario where people think that boring or expected results are 'failures' and don't bother to report on those, until somebody by random luck gets a misleading exciting result and reports on that."

"There's a whole dramatic debate about whether or not, in principle, we ought to pay any attention to results that aren't preregistered, if somebody reports on them anyways.  Even though that basically never happens in real life.  Because of how, if you say we should throw the results like that away, it means we're virtuously refusing to update on what is in fact evidence.  Conversely, if we pay attention to results like that, we're creating potentially bad systemic incentives."

"To the extent it ever did come up in real life, I figure people would just toss the original result and pay a 'replicator' to do it over again, if it was anything important."

"But, uh, I'm realizing as I say it, that none of this stuff is what we need to think about on a mini-Science! project the size of Project Lawful.  We're small enough that we can just know about all the experiments everyone is doing, and say all our individual probabilities on them out loud in advance.  We don't have professional replicators to appeal to, all we can do basically is run any important experiments twice with a different researcher in charge."

"I'm sort of flailing here, going through things in random order, as you can probably tell.  In dath ilan we all sort of grow up knowing how Science! is supposed to work, if only vaguely to start with, so at no point does anybody try to teach you the whole thing all at once.  There's just little pieces here and there where you learn in more detail how some bit of Science! works."

"So another precaution they emphasized was 'preregistration', you describe the experiment before you perform it and open a prediction market on the results before they're in.  That forces Civilization to make its own prediction in advance.  It lets anybody object beforehand instead of afterwards, if they have an issue with your experimental procedure.  It makes them look less persuasive if they wait until you get a result they don't like, to claim something was wrong with your methodology.  On a Civilizational level, we were told that 'preregistration' guards against a scenario where people think that boring or expected results are 'failures' and don't bother to report on those, until somebody by random luck gets a misleading exciting result and reports on that."

"There's a whole dramatic debate about whether or not, in principle, we ought to pay any attention to results that aren't 'preregistered', if somebody reports on them anyways.  Because of how, if you say we should throw the results like that away, it means we're virtuously refusing to update on what is in fact evidence.  Conversely, if we pay attention to results like that, we're creating bad systemic incentives."

"Obviously, that scenario basically never comes up in real life.  To the extent it ever did come up in real life, I figure people would just toss the original result and pay a 'replicator' to do it over again, if it was anything important."

"But, uh, I'm realizing as I say it, that none of this stuff is what we need to think about on a mini-Science! project the size of Project Lawful.  We're small enough that we can just know about all the experiments everyone is doing, and say all our individual probabilities on them out loud in advance.  We don't have professional replicators to appeal to, all we can do basically is run any important experiments twice with a different researcher in charge."

"I'm sort of flailing here, going through things in random order, as you can probably tell.  In dath ilan we all sort of grow up knowing how Science! is supposed to work, if only vaguely to start with, so at no point does anybody try to teach you the whole thing all at once.  There's just little pieces here and there where you learn in more detail how some bit of Science! works."