"Point #1, a 'published-experimental-report' wouldn't make up 'priors' or calculate a 'posterior' the way Asmodia did in her example.   That would be a weird kind of thing to claim as the knowledge of Civilization!  Somebody looking at your report might've seen other evidence.  They could be working on a secret Governance project, and know about evidence you can't access at all.  They might be considering hypotheses you haven't thought of at all.  Or, just, if you don't have formal ways of assigning 'priors', they might have more life experience in guessing those."

"Making claims about what Civilization should believe in the light of an experiment is a prediction market's job, not the job of one experimenter who publishes a report."

"If there's very standard or very obvious particular hypotheses to consider, you might make people's lives easier by precomputing the likelihood of the data on each of those hypotheses.  But one of the big cautions is to consider that other people might have hypotheses you don't, which means you should try to report more details of the data than you already know to be necessary."

"Taking the 2-4-6 example.  Suppose that people just hadn't thought of the possibility that there was 'temporal-dependence' in the results, that they hadn't broken out of the mindset where they were expecting the Keltham-environment to be a timeless 'function' that always produced identical output measurements for identical inputs.  They did run across one duplicate input, but didn't notice it was a duplicate input when they recorded the 'NO' - maybe because two different researchers were querying the environment separately."

"Now think of how inconvenienced you'd be, if that data was reported with input-ordering information missing, where somebody reading through later, couldn't see which inputs had happened before each other, because you'd implicitly believed that the Keltham-environment was 'time-invariant' and the inputs were 'exchangeable' in the ordering."

"And think of how helpful it would be, if the person had instead happened to include 'timestamps' on each input, even if they didn't know it was important, so you could notice the Keltham-environment taking longer to answer on some inputs, even if they'd missed that themselves."

"If the experiment was on people, the full report includes pseudonyms you can use to contact any of the experimental subjects, just in case somebody comes along later and wants to test subjects to see if they've got a particular 'gene' in their heredity, say."

"Obviously, newspapers can't report all the data of an experiment in the middle of the text.  Even most people who read the 'published-experimental-report' directly won't review all the raw data themselves.  Summaries matter!  So yeah, the report will say, briefly up front and in more detail at the end, 'Here's the theoretical likelihoods of the data for the major hypotheses under consideration.'  The newspaper story will say that, plus, 'Here's how prediction markets on related observables shifted immediately after publication.'"

"But the real report, above all, is the data itself - what Reality answered back to you when you asked it a question."

"Point #1, a 'published-experimental-report' wouldn't make up 'priors' or calculate a 'posterior' the way Asmodia did in her example.   That would be a weird kind of thing to claim as the knowledge of Civilization!  Somebody looking at your report might've seen other evidence.  They could be working on a secret Governance project, and know about evidence you can't access at all.  They might be considering hypotheses you haven't thought of at all.  Or, just, if you don't have formal ways of assigning 'priors', they might have more life experience in guessing those."

"Making claims about what Civilization should believe in the light of an experiment is a prediction market's job, not the job of one experimenter who publishes a report."

"If there's very standard or very obvious particular hypotheses to consider, you might make people's lives easier by precomputing the likelihood of the data on each of those hypotheses.  But one of the big cautions is to consider that other people might have hypotheses you don't, which means you should try to report more details of the data than you already know to be necessary."

"Taking the 2-4-6 example.  Suppose that people just hadn't thought of the possibility that there was 'temporal-dependence' in the results, that they hadn't broken out of the mindset where they were expecting the Keltham-environment to be a timeless 'function' that always produced identical output measurements for identical inputs.  They did run across one duplicate input, but didn't notice it was a duplicate input when they recorded the 'NO' - maybe because two different researchers were querying the environment separately."

"Now think of how inconvenienced you'd be, if that data was reported with input-ordering information missing.  Somebody reading through later, couldn't see which inputs had happened before each other, because you'd implicitly believed that the Keltham-environment was 'time-invariant' and the inputs were 'exchangeable' in the ordering."

"And think of how helpful it would be, if the person had instead happened to include 'timestamps' on each input, even if they didn't know it was important, so you could notice the Keltham-environment taking longer to answer on some inputs, even if they'd missed that themselves."

"If the experiment was on people, the full report includes pseudonyms you can use to contact any of the experimental subjects, just in case somebody comes along later and wants to test subjects to see if they've got a particular 'gene' in their heredity, say."

"Obviously, newspapers can't report all the data of an experiment in the middle of the text.  Even most people who read the 'published-experimental-report' directly won't review all the raw data themselves.  Summaries matter!  So yeah, the report will say, briefly up front and in more detail at the end, 'Here's the theoretical likelihoods of the data for the major hypotheses under consideration.'  The newspaper story will say that, plus, 'Here's how prediction markets on related observables shifted immediately after the results came out.'"

"But the real report, above all, is the data itself - what Reality answered back to you when you asked it a question."