tag:blogger.com,1999:blog-6894866515532737257.post8120558417170811909..comments2024-04-22T21:33:32.590-07:00Comments on Probably Overthinking It: Bayes's theorem and logistic regressionAllen Downeyhttp://www.blogger.com/profile/01633071333405221858noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-6894866515532737257.post-32594185956365346362016-10-31T19:14:02.503-07:002016-10-31T19:14:02.503-07:00Prof Downey,
Thanks for the article. How did you ...Prof Downey, <br />Thanks for the article. How did you get these values:P(H|FF) = 0.26 after the second, and P(H|FFF) = 0.07 after the third.<br /><br />Thanks<br />Anonymoushttps://www.blogger.com/profile/10202239324994703634noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-56044562379207446382014-08-05T13:38:02.435-07:002014-08-05T13:38:02.435-07:00Don't worry about overposting, but as some poi...Don't worry about overposting, but as some point I might have to stop overreplying :)<br /><br />Reading between the lines, I think you are coming face to face with one of the central issues of Bayesian inference, which is how to interpret probabilities, and especially the prior probability.<br /><br />In this case, P(H) is the prior probability that I am in the right class. If I chose the classroom at random, P(H) would be low. But I am basing my solution on the assumption that I did not choose the classroom at random, but rather tried to go to the right place. And based on my prior experience with navigating unfamiliar campuses, I estimate that my chance of being in the right place is about 90%.<br /><br />In frequentist terms, you could say that the relevant sample space is "all the times I've tried to find the right room", rather than "all the classrooms on campus."<br /><br />In (subjective) Bayesian terms, you would say that 90% is my subjective degree of belief that I am in the right place, based on relevant background information.<br /><br />But I would not say (as I think you did) that I am making a claim about the university, or that my Downeyian university is very different from a real university. My analysis is based on a model and the simplifications that come with it, but I don't think the model is as weird as you suggest.<br /><br />Thanks for this line of questions; I think it is productive.Allen Downeyhttps://www.blogger.com/profile/01633071333405221858noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-11267127084434500202014-08-05T13:25:53.857-07:002014-08-05T13:25:53.857-07:00Many thanks for your replies, professor. I hope I&...Many thanks for your replies, professor. I hope I'm not overposting.<br /><br />Would it be fair to say that my misconception of the problem is that I'm taking Olin University as the population; whereas I should be considering the population not as the Olin University, but a "Downeyian University". <br /><br />The Downeyian University is a special university ... one in which you have a 90% chance of turning up to the right class ... and not the Olin University, in which you would have only a very small chance of turning up at the right class if you just chose one at random.<br /><br />And this Downeyian University is a very strange University indeed ... because although you can specify what classes you are likely to turn up correctly for (it's the ones that you teach), you don't know what classes constitute the incorrect choices. They will be some proper subset of the entire Olin University, but we don't know what. The only thing we can say about it is that there is the same proportion of males as females. That would presumably be an assumption.<br /><br />But there's more! Although you're assuming that proportion of incorrectly chosen, you might be wrong. In fact, it's even plausible. How? Well, suppose you mostly gives lectures in the science faculty. Suppose that the science students are 90% male - not 50% male - and exactly the same proportion as your own class. What happens then, of course, is that the presence of females would actually give you no information.<br /><br />And maybe the situation is even worse than that! Maybe the actual "Downeyian" population contains more than 90% males, but that the males have a disproportionately larger distaste for mathematics and programming. Maybe they prefer engineering, or something. In that case, your intuition would have to be entirely flipped around ... the presence of females would be a positive indication that you're actually in the right class.<br /><br />Or perhaps I've got the wrong end of the stick. But I think that what I'm saying makes sense.<br /><br />Who would have thought statistics could be so much fun? ;)Anonymoushttps://www.blogger.com/profile/04470463426170671630noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-35677542581454940972014-08-05T12:17:37.645-07:002014-08-05T12:17:37.645-07:00Ah, now I see the problem! My previous reply was ...Ah, now I see the problem! My previous reply was wrong, but the numbers in the article are correct (but explained badly).<br /><br />As you said, the denominator P(F) should be P(F|H) P(H) + P(F|-H) P(-H), which is 0.14, not 0.5, and that yields P(H|F) = 0.64.<br /><br />In your first message, you objected to this denominator because you said it assumes that my class makes up 90% of the population of students. I think that's not right -- rather it takes into account that I am initially 90% sure that I am in the right class. But the term P(F|H) = 0.5 assumes (as you suggest) that my class is an insignificant part of the student population.<br /><br />Sorry for my confusion, and thanks for pointing this out. When I have a chance, I will edit the article to clarify.Allen Downeyhttps://www.blogger.com/profile/01633071333405221858noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-34299538050538101042014-08-05T11:43:10.780-07:002014-08-05T11:43:10.780-07:00But if you take P(H) = 0.9, P(F|H) = 0.1, P(F) = 0...But if you take P(H) = 0.9, P(F|H) = 0.1, P(F) = 0.5 and plug it into the formula, you get<br />P(H|F) = P(H) * P(F|H) / P(F)<br />= 0.9 * 0.1 / 0.5 = 0.18<br />which is not the answer of 0.64 that you gave in your post.Anonymoushttps://www.blogger.com/profile/04470463426170671630noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-45907208063175877762014-08-05T11:31:27.135-07:002014-08-05T11:31:27.135-07:00I'm not positive I understand where you see a ...I'm not positive I understand where you see a problem, but I think I agree with you. P(F) is the probability of a female student regardless of H, so it should be the overall fraction of female students at the university, probably close to 0.5. That's what I used in my calculations.Allen Downeyhttps://www.blogger.com/profile/01633071333405221858noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-357144002469479162014-08-05T11:26:36.133-07:002014-08-05T11:26:36.133-07:00There seems something intuitively wrong with your ...There seems something intuitively wrong with your calculation of P(F). You are saying:<br />P(F) = P(F|H) P(H) + P(F|-H) P(-H)<br />= 0.1 * 0.9 + 0.5 * 0.1<br />= 0.14<br />and using P(H) = 0.9, P(F|H) = 0.1 to obtain<br />P(H|F) = P(H) * P(F|H) / P(F)<br />= 0.9 * 0.1 / 0.14 = 0.64<br /><br />But here's the problem: the university is large, and your class is relatively small. So, given that P(F) is the likelhood of the data INDEPENDENT of H, you would expect that the overall university ratio of females would swamp out any skewing that your classes might introduce. In other words, I would expect P(F) = 0.5 (approx), not 0.14.<br /><br />The problem appears to be that you are assuming that the students in your class constitute 90% of the population of students (i.e. P(H)) of the university, whereas in fact they are likely to constitute only a miniscule proportion. That's why there's a skewing.<br /><br />Comments?Anonymoushttps://www.blogger.com/profile/04470463426170671630noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-88241670245344860642014-04-29T05:19:22.096-07:002014-04-29T05:19:22.096-07:00Wow, this was interesting! While I don't have ...Wow, this was interesting! While I don't have a background in logistic regression (yet), this was a fantastic first exposure to the versatility and usefulness of Bayes's theorem.<br />I loved your example about the college students - we really do use statistics and probability in everyday situations.<br /><br />14288941Anonymoushttps://www.blogger.com/profile/07726737137879127653noreply@blogger.com