Keltham starts by considering the case where N balls go LEFT and 0 balls go RIGHT.
Then every point lying a fraction p along the way from the LEFT side to the RIGHT side, has a likelihood p^N of yielding the data we saw.
The prior probability is the same everywhere, so the posterior function will be proportional to p^N everywhere. If you wanted the whole thing to sum to 1, you'd divide every point by whatever p^N sums to between 0 and 1.
Now, before they try to prove a correct answer, they should, of course, first figure out what the correct answer is, so they can prove it.
The correct answer in this case obviously has to be that it sums to 1/(N+1).
Why? Well, you saw N balls go LEFT. On the first round, that had probability 1/2. On the second round, you thought there was a 2/3 chance of seeing LEFT, and then you saw it, so your joint probability over all predictions is now 1/2 * 2/3 = 1/3 after seeing two LEFT. On the third round, you think there's a 3/4 chance of seeing LEFT, so your joint probability goes to 1/2 * 2/3 * 3/4 = 1/4 after seeing three LEFT.
Since your joint probability of seeing N balls go LEFT is 1/(N+1), the sum from 0 to 1 of an infinite number of points each with weight p^N for how much they predicted seeing N balls going left, must clearly end up as 1/(N+1).
Keltham starts by considering the case where N balls go LEFT and 0 balls go RIGHT.
Then every point lying a fraction p along the way from the LEFT side to the RIGHT side, has a likelihood p^N of yielding the data we saw.
The prior probability is the same everywhere, so the posterior function will be proportional to p^N everywhere. If you wanted the whole thing to sum to 1, you'd divide every point by whatever p^N sums to between 0 and 1.
Now, before they try to prove a correct answer, they should, of course, first figure out what the correct answer is, so they can prove it.
The correct answer in this case obviously has to be that it sums to 1/(N+1).
Why? Well, you saw N balls go LEFT. On the first round, that had probability 1/2. On the second round, you thought there was a 2/3 chance of seeing LEFT, and then you saw it, so your joint probability over all predictions is now 1/2 * 2/3 = 1/3 after seeing two LEFT. On the third round, you think there's a 3/4 chance of seeing LEFT, so your joint probability goes to 1/2 * 2/3 * 3/4 = 1/4 after seeing three LEFT.
Since your joint probability of seeing N balls go LEFT is 1/(N+1), the sum from 0 to 1 of an infinite number of points each with weight p^N for how much they predicted seeing N balls going LEFT, must clearly end up as 1/(N+1).
Keltham starts by considering the case where N balls go LEFT and 0 balls go RIGHT.
Then every point lying a fraction p along the way from the LEFT side to the RIGHT side, has a likelihood p^N of yielding the data we saw.
Retroactive class chatter
One student is having some trouble parsing this, despite Keltham's gainful attempts to speak slowly. The p is a position along a line from 0 to 1, and the p^n is 'if the 0-ball landed at that point, how likely it is that it yielded the data we saw'. But p=0 at the far left, shouldn't it be infinitely likely it be to the left of the furthest-left point instead of no chance whatsoever -
A moment later, they've worked out that, no, the next ball has probability 0 of being any further left than as far left as possible, and the whole thing flips around in their head.
But by then, of course, Keltham has already continued talking.
The prior probability is the same everywhere, so the posterior function will be proportional to p^N everywhere. If you wanted the whole thing to sum to 1, you'd divide every point by whatever p^N sums to between 0 and 1.
Retroactive Alexandre intertag
Alexandre was managing to keep up until this point; his brain spent a fractional second thinking that if p was at the far left side, then the next ball to roll was infinitely likely to be to the left of that, but only a fractional second and it soon flipped around to seeing that yes the next ball would have 0 probability of being further left.
"... Excuse me." Alexandre raises his hand, since showing weakness in this environment appears to be less bad than possessing it. 'Divide every point by whatever p^N sums to between 0 and 1' - can I ask you to expand on that?"
Keltham shall be happy to do so! Thank you for providing any feedback on where anyone is getting lost.
As a simple example, take the case of N=1 and p^N=p. That's just a triangle filling half a square with side 1, so obviously the sum 1/INF * [0/INF + 1/INF + 2/INF + ... + INF/INF] = 1/2 in the end.
Then to make the whole expression sum to 1 instead, they divide the whole thing by 1/2 or multiply it by 2. 1/INF * [0/INF + 1/INF + ... + INF/INF] / (1/2) = 1.
As Keltham is saying this, it occurs to him that it may not be obvious to the candidates why you'd do that, depending on what got covered in previous Probability classes and how well. He'll expand on that too.
If they were just tracking, say, 1/2 prior probability of propensity 1/3 versus 1/2 prior probability of propensity 2/3, the prior-times-likelihood object would look like {1/2 * 1/3, 1/2 * 2/3} and the sum over the prior-times-likelihood object would look like {1/6 + 1/3} = 1/2. Which is just what it should be, because if you start with 50-50 on 1/3 vs. 2/3 then your prior probability of seeing the ball go LEFT is indeed 1/2.
Conversely, the prior-time-likelihood object for seeing RIGHT is {1/2 * (1 - 1/3), 1/2 * (1 - 2/3)} = {1/2 * 2/3, 1/2 * 1/3} = {1/3, 1/6} again summing to 1/2.
This should look like it makes lots of sense. On priors, you think either the ball goes left, or the ball goes right, but not both, nor neither, so your prior probability of LEFT plus your probability of RIGHT should sum to 1.
But after actually seeing the ball go LEFT and updating on that, you throw out all the possible worlds from your probability distribution where the ball went RIGHT, and live only inside the worlds where the ball went LEFT. After seeing LEFT, then, within this kind of hypothesis where you're not questioning the whole setup from outside, you're certain that the ball went LEFT. It's now probability 1 that the ball went LEFT. The probability 1, of what is real, is divided up among the surviving worlds; the surviving worlds should have probability proportional to their prior times their likelihood, but summing to 1.
So you divide through {1/6, 1/3} by its total probability of 1/2 to get {1/3, 2/3} as the posterior probability.
So to get the posterior probability on all the hypotheses between 0 and 1, after seeing N balls go left, they take the prior-times-likelihood object, which is 1/INF * [ (0/INF)^N + (1/INF)^N + ... + (INF/INF)^N ], and divide it through by its sum so that the probability inside the whole thing sums to 1. If we started out certain that the ball has a propensity between 0 and 1 to go LEFT or RIGHT, and then we observe N LEFTs, we end up still certain that the ball has some propensity between 0 and 1 to go LEFT.
So the question then becomes: 1/INF * [ (0/INF)^N + (1/INF)^N + ... + (INF/INF)^N ] = ?
Now, before they try to prove a correct answer, they should, of course, first figure out what the correct answer is, so they can prove it.
The correct answer in this case obviously has to be that it sums to 1/(N+1).
Why? Well, you saw N balls go LEFT. On the first round, that had probability 1/2. On the second round, you thought there was a 2/3 chance of seeing LEFT, and then you saw it, so your joint probability over all predictions is now 1/2 * 2/3 = 1/3 after seeing two LEFT. On the third round, you think there's a 3/4 chance of seeing LEFT, so your joint probability goes to 1/2 * 2/3 * 3/4 = 1/4 after seeing three LEFT.
Since your joint probability of seeing N balls go LEFT is 1/(N+1), the sum from 0 to 1 of an infinite number of points each with weight p^N for how much they predicted seeing N balls going LEFT, must clearly end up as 1/(N+1).
Keltham starts by considering the case where N balls go LEFT and 0 balls go RIGHT.
Then every point lying a fraction p along the way from the LEFT side to the RIGHT side, has a likelihood p^N of yielding the data we saw.
Retroactive class chatter
One student is having some trouble parsing this, despite Keltham's gainful attempts to speak slowly. The p is a position along a line from 0 to 1, and the p^n is 'if the 0-ball landed at that point, how likely it is that it yielded the data we saw'. But p=0 at the far left, shouldn't it be infinitely likely it be to the left of the furthest-left point instead of no chance whatsoever -
A moment later, they've worked out that, no, the next ball has probability 0 of being any further left than as far left as possible, and the whole thing flips around in their head.
But by then, of course, Keltham has already continued talking.
The prior probability is the same everywhere, so the posterior function will be proportional to p^N everywhere. If you wanted the whole thing to sum to 1, you'd divide every point by whatever p^N sums to between 0 and 1.
Retroactive Alexandre intertag
Alexandre was managing to keep up until this point; his brain spent a fractional second thinking that if p was at the far left side, then the next ball to roll was infinitely likely to be to the left of that, but only a fractional second and it soon flipped around to seeing that yes the next ball would have 0 probability of being further left.
"... Excuse me." Alexandre raises his hand, since showing weakness in this environment appears to be less bad than possessing it. 'Divide every point by whatever p^N sums to between 0 and 1' - can I ask you to expand on that?"
Keltham shall be happy to do so! Thank you for providing any feedback on where anyone is getting lost.
As a simple example, take the case of N=1 and p^N=p. That's just a triangle filling half a square with side 1, so obviously the sum 1/INF * [0/INF + 1/INF + 2/INF + ... + INF/INF] = 1/2 in the end.
Then to make the whole expression sum to 1 instead, they divide the whole thing by 1/2 or multiply it by 2. 1/INF * [0/INF + 1/INF + ... + INF/INF] / (1/2) = 1.
As Keltham is saying this, it occurs to him that it may not be obvious to the candidates why you'd do that, depending on what got covered in previous Probability classes and how well. He'll expand on that too.
If they were just tracking, say, 1/2 prior probability of propensity 1/3 versus 1/2 prior probability of propensity 2/3, the prior-times-likelihood object would look like {1/2 * 1/3, 1/2 * 2/3} and the sum over the prior-times-likelihood object would look like {1/6 + 1/3} = 1/2. Which is just what it should be, because if you start with 50-50 on 1/3 vs. 2/3 then your prior probability of seeing the ball go LEFT is indeed 1/2.
Conversely, the prior-time-likelihood object for seeing RIGHT is {1/2 * (1 - 1/3), 1/2 * (1 - 2/3)} = {1/2 * 2/3, 1/2 * 1/3} = {1/3, 1/6} again summing to 1/2.
This should look like it makes lots of sense. On priors, you think either the ball goes left, or the ball goes right, but not both, nor neither, so your prior probability of LEFT plus your probability of RIGHT should sum to 1.
But after actually seeing the ball go LEFT and updating on that, you throw out all the possible worlds from your probability distribution where the ball went RIGHT, and live only inside the worlds where the ball went LEFT. After seeing LEFT, then, within this kind of hypothesis where you're not questioning the whole setup from outside, you're certain that the ball went LEFT. It's now probability 1 that the ball went LEFT. The probability 1, of what is real, is divided up among the surviving worlds; the surviving worlds should have probability proportional to their prior times their likelihood, but summing to 1.
So you divide through {1/6, 1/3} by its total probability of 1/2 to get {1/3, 2/3} as the posterior probability.
So to get the posterior probability on all the hypotheses between 0 and 1, after seeing N balls go left, they take the prior-times-likelihood object, which is 1/INF * [ (0/INF)^N + (1/INF)^N + ... + (INF/INF)^N ], and divide it through by its sum so that the probability inside the whole thing sums to 1. If we started out certain that the ball has a propensity between 0 and 1 to go LEFT or RIGHT, and then we observe N LEFTs, we end up still certain that the ball has some propensity between 0 and 1 to go LEFT.
So the question then becomes: 1/INF * [ (0/INF)^N + (1/INF)^N + ... + (INF/INF)^N ] = ?
Now, before they try to prove a correct answer, they should, of course, first figure out what the correct answer is, so they can prove it.
The correct answer in this case obviously has to be that it sums to 1/(N+1).
Why? Well, you saw N balls go LEFT. On the first round, that had probability 1/2. On the second round, you thought there was a 2/3 chance of seeing LEFT, and then you saw it, so your joint probability over all predictions is now 1/2 * 2/3 = 1/3 after seeing two LEFT. On the third round, you think there's a 3/4 chance of seeing LEFT, so your joint probability goes to 1/2 * 2/3 * 3/4 = 1/4 after seeing three LEFT.
Since your joint probability of seeing N balls go LEFT is 1/(N+1), the sum from 0 to 1 of an infinite number of points each with weight p^N for how much they predicted seeing N balls going LEFT, must clearly end up as 1/(N+1).
Keltham starts by considering the case where N balls go LEFT and 0 balls go RIGHT.
Then every point lying a fraction p along the way from the LEFT side to the RIGHT side, has a likelihood p^N of yielding the data we saw.
Retroactive class chatter
One student is having some trouble parsing this, despite Keltham's gainful attempts to speak slowly. The p is a position along a line from 0 to 1, and the p^n is 'if the 0-ball landed at that point, how likely it is that it yielded the data we saw'. But p=0 at the far left, shouldn't it be infinitely likely it be to the left of the furthest-left point instead of no chance whatsoever -
A moment later, they've worked out that, no, the next ball has probability 0 of being any further left than as far left as possible, and the whole thing flips around in their head.
But by then, of course, Keltham has already continued talking.
The prior probability is the same everywhere, so the posterior function will be proportional to p^N everywhere. If you wanted the whole thing to sum to 1, you'd divide every point by whatever p^N sums to between 0 and 1.
Retroactive Alexandre intertag
"... Excuse me." Alexandre raises his hand, since showing weakness in this environment appears to be less bad than possessing it. 'Divide every point by whatever p^N sums to between 0 and 1' - can I ask you to expand on that?"
Keltham shall be happy to do so! Thank you for providing any feedback on where anyone is getting lost.
As a simple example, take the case of N=1, so p^N=p. If you graph that, you get a straight line from likelihood=0 at p=0, to likelihood 1 at p=1. That graph looks like a triangle filling half of a square with side 1. So by visual reasoning, the sum 1/INF * [0/INF + 1/INF + 2/INF + ... + INF/INF] = 1/2 in the end.
Proof by dubious infinitary mathematics, now that they know the correct answer by simpler visual reasoning:
Then to find a probability distribution proportional to that prior-times-likelihood object, but which sums to 1 to form the posterior, they divide the whole thing by 1/2 or multiply it by 2.
As Keltham is saying this, it occurs to him that it may not be obvious to the candidates why you'd do that, depending on what got covered in previous Probability classes and how well. He'll expand on that too.
If they were just tracking, say, 1/2 prior probability of propensity 1/3 versus 1/2 prior probability of propensity 2/3, the prior-times-likelihood object would look like {1/2 * 1/3, 1/2 * 2/3} and the sum over the prior-times-likelihood object would look like {1/6 + 1/3} = 1/2. Which is just what it should be, because if you start with 50-50 on 1/3 vs. 2/3 then your prior probability of seeing the ball go LEFT is indeed 1/2.
Conversely, the prior-time-likelihood object for seeing RIGHT is {1/2 * (1 - 1/3), 1/2 * (1 - 2/3)} = {1/2 * 2/3, 1/2 * 1/3} = {1/3, 1/6} again summing to 1/2.
This should look like it makes lots of sense. On priors, you think either the ball goes left, or the ball goes right, but not both, nor neither, so your prior probability of LEFT plus your probability of RIGHT should sum to 1.
But after actually seeing the ball go LEFT and updating on that, you throw out all the possible worlds from your probability distribution where the ball went RIGHT, and live only inside the worlds where the ball went LEFT. After seeing LEFT, then, within this kind of hypothesis where you're not questioning the whole setup from outside, you're certain that the ball went LEFT. It's now probability 1 that the ball went LEFT. The probability 1, of what is real, is divided up among the surviving worlds; the surviving worlds should have probability proportional to their prior times their likelihood, but summing to 1.
So you divide through {1/6, 1/3} by its total probability of 1/2 to get {1/3, 2/3} as the posterior probability.
So to get the posterior probability on all the hypotheses between 0 and 1, after seeing N balls go left, they take the prior-times-likelihood object, which is 1/INF * [ (0/INF)^N + (1/INF)^N + ... + (INF/INF)^N ], and divide it through by its sum so that the probability inside the whole thing sums to 1. If we started out certain that the ball has a propensity between 0 and 1 to go LEFT or RIGHT, and then we observe N LEFTs, we end up still certain that the ball has some propensity between 0 and 1 to go LEFT.
So the question then becomes: 1/INF * [ (0/INF)^N + (1/INF)^N + ... + (INF/INF)^N ] = ?
Now, before they try to prove a correct answer, they should, of course, first figure out what the correct answer is, so they can prove it.
The correct answer in this case obviously has to be that it sums to 1/(N+1).
Why? Well, you saw N balls go LEFT. On the first round, that had probability 1/2. On the second round, you thought there was a 2/3 chance of seeing LEFT, and then you saw it, so your joint probability over all predictions is now 1/2 * 2/3 = 1/3 after seeing two LEFT. On the third round, you think there's a 3/4 chance of seeing LEFT, so your joint probability goes to 1/2 * 2/3 * 3/4 = 1/4 after seeing three LEFT.
Since your joint probability of seeing N balls go LEFT is 1/(N+1), the sum from 0 to 1 of an infinite number of points each with weight p^N for how much they predicted seeing N balls going LEFT, must clearly end up as 1/(N+1).
Keltham starts by considering the case where N balls go LEFT and 0 balls go RIGHT.
Then every point lying a fraction p along the way from the LEFT side to the RIGHT side, has a likelihood p^N of yielding the data we saw.
Retroactive parenthetical class chatter
One student is having some trouble parsing this, despite Keltham's gainful attempts to speak slowly. The p is a position along a line from 0 to 1, and the p^n is 'if the 0-ball landed at that point, how likely it is that it yielded the data we saw'. But p=0 at the far left, shouldn't it be infinitely likely it be to the left of the furthest-left point instead of no chance whatsoever -
A moment later, they've worked out that, no, the next ball has probability 0 of being any further left than as far left as possible, and the whole thing flips around in their head.
But by then, of course, Keltham has already continued talking.
The prior probability is the same everywhere, so the posterior function will be proportional to p^N everywhere. If you wanted the whole thing to sum to 1, you'd divide every point by whatever p^N sums to between 0 and 1.
Retroactive parenthetical Alexandre intertag
"... Excuse me." Alexandre raises his hand, since showing weakness in this environment appears to be less bad than possessing it. 'Divide every point by whatever p^N sums to between 0 and 1' - can I ask you to expand on that?"
Keltham shall be happy to do so! Thank you for providing any feedback on where anyone is getting lost.
As a simple example, take the case of N=1, so p^N=p. If you graph that, you get a straight line from likelihood=0 at p=0, to likelihood 1 at p=1. That graph looks like a triangle filling half of a square with side 1. So by visual reasoning, the sum 1/INF * [0/INF + 1/INF + 2/INF + ... + INF/INF] = 1/2 in the end.
Proof by dubious infinitary mathematics, now that they know the correct answer by simpler visual reasoning:
Then to find a probability distribution proportional to that prior-times-likelihood object, but which sums to 1 to form the posterior, they divide the whole thing by 1/2 or multiply it by 2.
As Keltham is saying this, it occurs to him that it may not be obvious to the candidates why you'd do that, depending on what got covered in previous Probability classes and how well. He'll expand on that too.
If they were just tracking, say, 1/2 prior probability of propensity 1/3 versus 1/2 prior probability of propensity 2/3, the prior-times-likelihood object would look like {1/2 * 1/3, 1/2 * 2/3} and the sum over the prior-times-likelihood object would look like {1/6 + 1/3} = 1/2. Which is just what it should be, because if you start with 50-50 on 1/3 vs. 2/3 then your prior probability of seeing the ball go LEFT is indeed 1/2.
Conversely, the prior-time-likelihood object for seeing RIGHT is {1/2 * (1 - 1/3), 1/2 * (1 - 2/3)} = {1/2 * 2/3, 1/2 * 1/3} = {1/3, 1/6} again summing to 1/2.
This should look like it makes lots of sense. On priors, you think either the ball goes left, or the ball goes right, but not both, nor neither, so your prior probability of LEFT plus your probability of RIGHT should sum to 1.
But after actually seeing the ball go LEFT and updating on that, you throw out all the possible worlds from your probability distribution where the ball went RIGHT, and live only inside the worlds where the ball went LEFT. After seeing LEFT, then, within this kind of hypothesis where you're not questioning the whole setup from outside, you're certain that the ball went LEFT. It's now probability 1 that the ball went LEFT. The probability 1, of what is real, is divided up among the surviving worlds; the surviving worlds should have probability proportional to their prior times their likelihood, but summing to 1.
So you divide through {1/6, 1/3} by its total probability of 1/2 to get {1/3, 2/3} as the posterior probability.
So to get the posterior probability on all the hypotheses between 0 and 1, after seeing N balls go left, they take the prior-times-likelihood object, which is 1/INF * [ (0/INF)^N + (1/INF)^N + ... + (INF/INF)^N ], and divide it through by its sum so that the probability inside the whole thing sums to 1. If we started out certain that the ball has a propensity between 0 and 1 to go LEFT or RIGHT, and then we observe N LEFTs, we end up still certain that the ball has some propensity between 0 and 1 to go LEFT.
So the question then becomes: 1/INF * [ (0/INF)^N + (1/INF)^N + ... + (INF/INF)^N ] = ?
Now, before they try to prove a correct answer, they should, of course, first figure out what the correct answer is, so they can prove it.
The correct answer in this case obviously has to be that it sums to 1/(N+1).
Why? Well, you saw N balls go LEFT. On the first round, that had probability 1/2. On the second round, you thought there was a 2/3 chance of seeing LEFT, and then you saw it, so your joint probability over all predictions is now 1/2 * 2/3 = 1/3 after seeing two LEFT. On the third round, you think there's a 3/4 chance of seeing LEFT, so your joint probability goes to 1/2 * 2/3 * 3/4 = 1/4 after seeing three LEFT.
Since your joint probability of seeing N balls go LEFT is 1/(N+1), the sum from 0 to 1 of an infinite number of points each with weight p^N for how much they predicted seeing N balls going LEFT, must clearly end up as 1/(N+1).
