The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" it is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/5! = 1!*4!/5! = 24/120 = obviously 1/5 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7 instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/5 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/25 vs. 1/1024, and 41:1 isn't enough to make you stop believing a coin is fair. Fewer than 1/42 coins are fixed to start with, even in Golarion.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:25 ~ 111:1 that they were two separate sources, after that. So if that was at all plausible to start with, you're pretty darned suspicious even after 5 samples apiece, if the resulting ratios are extreme. (Again, it's different if you start out with an even stronger suspicion that something is a fair coin, rather than wondering if 2 things have 2 unknown propensities or 1 unknown propensity.)
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the probability.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources", for the overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate an effect size". No, not even in worlds where all you do to combine the analyses from two experiments is just multiply the likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two hypotheses say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" it is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection, and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/5! = 1!*4!/5! = 24/120 = obviously 1/5 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7 instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/5 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/25 vs. 1/1024, and 41:1 isn't enough to make you stop believing a coin is fair. Fewer than 1/42 coins are fixed to start with, even in Golarion.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:25 ~ 111:1 that they were two separate sources, after that. So if that was at all plausible to start with, you're pretty darned suspicious even after 5 samples apiece, if the resulting ratios are extreme. (Again, it's different if you start out with an even stronger suspicion that something is a fair coin, rather than wondering if 2 things have 2 unknown propensities or 1 unknown propensity.)
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the probability.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources", for the overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate an effect size". No, not even in worlds where all you do to combine the analyses from two experiments is just multiply the likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two hypotheses say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" it is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection, and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/5! = 1!*4!/5! = 24/120 = obviously 1/5 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7 instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/5 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/25 vs. 1/1024, and 41:1 isn't enough to make you stop believing a coin is fair. Fewer than 1/42 coins are fixed to start with, even in Golarion.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:25 ~ 111:1 that they were two separate sources, after that. So if that was at all plausible to start with, you're pretty darned suspicious even after 5 samples apiece, if the resulting ratios are extreme. (Again, it's different if you start out with an even stronger suspicion that something is a fair coin, rather than wondering if 2 things have 2 unknown propensities or 1 unknown propensity.)
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the probability.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources". The overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate an effect size". No, not even in worlds where all you do to combine the analyses from two experiments is just multiply the likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two hypotheses say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" it is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection, and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/6! = 1!*4!/6! = 24/720 = obviously 1/30 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus over a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7 instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/5 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/900 vs. 1/1024.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:900 ~ 3:1 that they were two separate sources, after that. If it wasn't pretty plausible to start, it's not plausible yet now, after so little data. If you were already pretty suspicious, you're now quite noticeably more suspicious, though.
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the probability.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources". The overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate an effect size". No, not even in worlds where all you do to combine the analyses from two experiments is just multiply the likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two hypotheses say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" it is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection, and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/6! = 1!*4!/6! = 24/720 = obviously 1/30 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus over a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7 instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/5 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/900 vs. 1/1024.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:900 ~ 3:1 that they were two separate sources, after that. If it wasn't pretty plausible to start, it's not plausible yet now, after so little data. If you were already pretty suspicious, you're now quite noticeably more suspicious, though.
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the probability.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources". The overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate an effect size". No, not even in worlds where all you do to combine the analyses from two experiments is just multiply the likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two hypotheses say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
It's just one of those things that, like, is incredibly important to doing real Science! and jumps directly out at you, once you start thinking about experimental reports in terms of likelihoods, which, in fact, the median dath ilani will do even if nobody explicitly tells them so and even if their entire world tries to tell them otherwise.
The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" it is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection, and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/6! = 1!*4!/6! = 24/720 = obviously 1/30 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus over a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7 instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/5 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/900 vs. 1/1024.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:900 ~ 3:1 that they were two separate sources, after that. If it wasn't pretty plausible to start, it's not plausible yet now, after so little data. If you were already pretty suspicious, you're now quite noticeably more suspicious, though.
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the probability.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources". The overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate the parameter". No, not even in sane worlds where all you do to combine the analyses from two experiments is just multiply their two likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two datasets say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
It's just one of those things that, like, is incredibly important to doing real Science! and jumps directly out at you, once you start thinking about experimental reports in terms of likelihoods, which, in fact, the median dath ilani will do even if nobody explicitly tells them so and even if their entire world tries to tell them otherwise.
The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" it is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection, and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/6! = 1!*4!/6! = 24/720 = obviously 1/30 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus over a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7 instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/30 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/900 vs. 1/1024.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:900 ~ 3:1 that they were two separate sources, after that. If it wasn't pretty plausible to start, it's not plausible yet now, after so little data. If you were already pretty suspicious, you're now quite noticeably more suspicious, though.
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the probability.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources". The overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate the parameter". No, not even in sane worlds where all you do to combine the analyses from two experiments is just multiply their two likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two datasets say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
It's just one of those things that, like, is incredibly important to doing real Science! and jumps directly out at you, once you start thinking about experimental reports in terms of likelihoods, which, in fact, the median dath ilani will do even if nobody explicitly tells them so and even if their entire world tries to tell them otherwise.
The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection, and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/6! = 1!*4!/6! = 24/720 = obviously 1/30 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus over a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7 instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/30 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/900 vs. 1/1024.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:900 ~ 3:1 that they were two separate sources, after that. If it wasn't pretty plausible to start, it's not plausible yet now, after so little data. If you were already pretty suspicious, you're now quite noticeably more suspicious, though.
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the probability.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources". The overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate the parameter". No, not even in sane worlds where all you do to combine the analyses from two experiments is just multiply their two likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two datasets say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
It's just one of those things that, like, is incredibly important to doing real Science! and jumps directly out at you, once you start thinking about experimental reports in terms of likelihoods, which, in fact, the median dath ilani will do even if nobody explicitly tells them so and even if their entire world tries to tell them otherwise.
The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection, and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/6! = 1!*4!/6! = 24/720 = obviously 1/30 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus over a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7 instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/30 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/900 vs. 1/1024.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:900 ~ 3:1 that they were two separate sources, after that. If it wasn't pretty plausible to start, it's not plausible yet now, after so little data. If you were already pretty suspicious, you're now quite noticeably more suspicious, though.
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the likelihood density.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources". The overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate the parameter". No, not even in sane worlds where all you do to combine the analyses from two experiments is just multiply their two likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two datasets say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
It's just one of those things that, like, is incredibly important to doing real Science! and jumps directly out at you, once you start thinking about experimental reports in terms of likelihoods, which, in fact, the median dath ilani will do even if nobody explicitly tells them so and even if their entire world tries to tell them otherwise.
The median dath ilani has - by Golarion's standards - some power that Golarion knows not, or aptitude that it knows very little. It isn't easily captured by the INT score that is measured by Detect Thoughts. Even if you add in whatever "Wisdom" is measured by Detect Anxieties, there's some residual that isn't measured still.
It's not just about the training the dath ilani undergo in childhood. No, really, it's not. They have been doing a lot of mate-selection, and not just since there've been prediction markets about what kind of children would result.
If somehow you took the median dath ilani, and exposed them to only a few fragments of Law such as Keltham has already taught, a few stories about cognitive science such as Keltham has already told, if they'd had only that little true education by the time they'd reached adulthood -
- the median dath ilani would still be told of the Law of Succession, and think spontaneously, without any outside prompting, of how you could use that to guess whether two sources were the same or different. Just look at the scores if you use the Law of Succession twice, separately, versus using the Law of Succession on both of them together. The difference between those scores is the likelihood ratio for different versus same.
If you get 4 LEFT and 1 RIGHT off one source, and 4 RIGHT and 1 LEFT off another source, then analyzing them separately gives you scores for each of 4!*1!/6! = 1!*4!/6! = 24/720 = obviously 1/30 just look at the factorials. Analyzing both datasets together gives you 5!*5!/11!, which you can see intuitively is going to end up around 1/1024, minus over a bit for the product being 1/2 * 1/3 * 2/4 * 2/5 * 3/6 * 3/7... instead of just 1/2^(10). (It's actually 1/2772 if you bother to calculate - not as nightmarish as it looks, you can cancel a lot of factors.)
Point is, you've got a set of two likelihood functions for two separate datasets, versus one likelihood function on the combined dataset, where, if you started with a uniform prior on three propensity spaces, and multiplied by all those likelihood functions, the two separate functions would destroy all but 1/30 of the probability on each of the two separate spaces, and the combined likelihood function would destroy all but 1/2772 of the uniform probability on its own parameter space.
It's not to the point where if you literally pulled a coin and flipped it 5 times twice, you'd conclude shenanigans from seeing 4 LEFT and 1 RIGHT the first time, versus 1 RIGHT and 4 LEFT the second time. That's just a likelihood ratio of 1/900 vs. 1/1024.
But if the problem is more mysterious than that? If you are less certain at the start that your data is coming from a single source across both cases? Then you'd be looking at an update of more like 2772:900 ~ 3:1 that they were two separate sources, after that. If it wasn't pretty plausible to start, it's not plausible yet now, after so little data. If you were already pretty suspicious, you're now quite noticeably more suspicious, though.
The median dath ilani - even given only such education as Keltham has already provided - fewer hints than that, even - would spontaneously generalize the principle of taking alarm if two experiments seemed to have nonoverlapping likelihood functions.
Suppose the likelihood functions are over a simple hypothesis space - such that likelihood functions form clouds naturally visible in that space - such that there is a natural way to informally see boundaries around narrow subvolumes of the clouds.
You can't say it in an absolute way, apart from some prior and arbitrary concept of how to draw boundaries like that and divide up the space. You could always throw some random points into an otherwise compact cloud and say that you thought they should be in there.
But informally, it's natural enough to see 376 LEFTs and 624 RIGHTs, look at the likelihood function P(data|propensity=p) = p^376*(1-p)^624, and say, "That cloud has 90% of its density between p=35% and p=40%."
If you widen to the amount between p=30% and p=45%, that's 99.9999% of the likelihood density.
And then let's say that you run a different experiment, and it turns up 602 LEFTs and 398 RIGHTs.
Informally - for there is no way to say it formally, without introducing an arbitrary note of subjectivity; we are looking to cues that the data gives us to look outside our hypothesis space, and there is a limit to how much you can ever formalize that, without invoking enough Law to create a mortal from scratch - informally, you look at that and say "No way in superheated toilet paper are those the same two data sources". The overlap of the two clouds in likelihood-space is virtually zero. They have each eliminated practically all of the probability from any hypothesis that could non-stupidly account for the other, and those two different stories cannot exist in the same world.
You do not "combine the data from the two experiments to estimate the parameter". No, not even in sane worlds where all you do to combine the analyses from two experiments is just multiply their two likelihood functions together. Sure, you can multiply p^376*(1-p)^624 by p^602*(1-p)^398 and get p^978*(1-p)^1022, but you obviously shouldn't do that. The first trial concentrates 99.9999% of its survivability between propensities of 0.30 and 0.45, and the second concentrates 99.9995% of its survivability between 0.53 and 0.67. There's no overlap between what the two datasets say are the livable regions of the parameter space.
These two experiments were not conducted with the same world feeding them their answers, though, ultimately, they were conducted inside the same greater Reality. Something is wrong in one place or both.
It's just one of those things that, like, is incredibly important to doing real Science! and jumps directly out at you, once you start thinking about experimental reports in terms of likelihoods, which, in fact, the median dath ilani will do even if nobody explicitly tells them so and even if their entire world tries to tell them otherwise.
