tag:blogger.com,1999:blog-6894866515532737257.post8330883106947449454..comments2024-04-22T21:33:32.590-07:00Comments on Probably Overthinking It: Bayesian solution to the Lincoln index problemAllen Downeyhttp://www.blogger.com/profile/01633071333405221858noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-6894866515532737257.post-59880230487550199632014-08-04T13:50:03.505-07:002014-08-04T13:50:03.505-07:00Just an extra note: I'm learning a bit of Stan...Just an extra note: I'm learning a bit of Stan and I coded the exact same model as in BUGS. The result after 100k iterations was 72.11. So, another confirmation :-) By some mysterious reason BUGS is having problems dealing with this likelihood function.João Netohttps://www.blogger.com/profile/05560718055133816500noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-38985412746039363392014-07-23T07:15:56.542-07:002014-07-23T07:15:56.542-07:00Well, not 'where' but 'how' :-) I ...Well, not 'where' but 'how' :-) I used Mathematica to make the integration and R to find the mode. The details are at the end of the webpage I made.<br /><br />Cheers,João Netohttps://www.blogger.com/profile/05560718055133816500noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-82580042191608181412014-07-23T07:08:09.613-07:002014-07-23T07:08:09.613-07:00Huh. Well, let me know if you figure it out.
Whe...Huh. Well, let me know if you figure it out.<br /><br />Where did you find the analytic solution?Allen Downeyhttps://www.blogger.com/profile/01633071333405221858noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-76711349270150731902014-07-19T10:55:22.346-07:002014-07-19T10:55:22.346-07:00I found the analytic solution for p(n|data) and th...I found the analytic solution for p(n|data) and the mode is at 72.18 which confirms your result. My result for the mode with the uniform prior is at 75.9. I really don't know what is causing this difference but I suppose it is from my BUGS model. Oh well...João Netohttps://www.blogger.com/profile/05560718055133816500noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-79206650998399083122014-07-18T11:52:13.861-07:002014-07-18T11:52:13.861-07:00Yes, good point. Thanks!Yes, good point. Thanks!Allen Downeyhttps://www.blogger.com/profile/01633071333405221858noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-22451905110809727862014-07-18T11:27:50.159-07:002014-07-18T11:27:50.159-07:00I think it should be noted that the assumption tha...I think it should be noted that the assumption that all error probabilities are the same makes a big difference. Suppose I have a program with 100 bugs, 5 of which are easy to find, and two testers who are bad at finding bugs. Tester 1 finds 10 bugs (including all 5 easy ones), tester 2 finds 10 bugs (including all 5 easy ones), and the intersection is the 5 easy ones. The Lincoln index would estimate 20 bugs and your method would find something similar (I'm assuming, I didn't implement the code). In general, the more the probabilities differ from error to error, the more these methods will underestimate the total number of errors. They do work quite well, though, when error probabilities are uniform. Bryanhttps://www.blogger.com/profile/15975987326012523061noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-35114467506693942042014-07-15T08:55:18.324-07:002014-07-15T08:55:18.324-07:00Much better than the Cooke post. Cooke's post...Much better than the Cooke post. Cooke's post has a flaw - if you send an app to a million beta testers, and 5000 users find a few bugs, your bug count projection spirals high very quickly (since the numerator ultimately becomes an exponential function).<br /><br />I intend to give this a whirl and see what numbers I get.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-37603483965944803822014-07-11T07:37:03.607-07:002014-07-11T07:37:03.607-07:00Huh. I'm baffled for now. Let me know if you...Huh. I'm baffled for now. Let me know if you learn more.Allen Downeyhttps://www.blogger.com/profile/01633071333405221858noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-92080627314854601042014-07-11T06:51:40.269-07:002014-07-11T06:51:40.269-07:00With an uniform prior, the mean for 'n' is...With an uniform prior, the mean for 'n' is 108.9 and the MLE estimate is 78. I think this makes sense, since the uniform gives move probability mass to the upper values, relative to the exponential. However, this result produces even a bigger difference between our results (?)<br /><br />The simulation run took 2 seconds.<br /><br />I assume that it is a problem with my model, but in my defense the simulation I do at the end for the expected value of 'c' is very near 3 :-) With the exponential the simulated result was 2.98. With the uniform the result was 2.95.João Netohttps://www.blogger.com/profile/05560718055133816500noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-2108579856498488812014-07-11T06:30:08.563-07:002014-07-11T06:30:08.563-07:00That's excellent -- thanks!
My results are ba...That's excellent -- thanks!<br /><br />My results are based on a uniform prior for n, which was an arbitrary choice, not motivated by any background knowledge. That might explain the difference between your results and mine, but I agree with you that it seems like a big difference.<br /><br />Can you run yours again with a uniform prior on [32, 350]?<br /><br />Also, how long does your model take to run?Allen Downeyhttps://www.blogger.com/profile/01633071333405221858noreply@blogger.comtag:blogger.com,1999:blog-6894866515532737257.post-65199642275224221332014-07-11T05:48:08.128-07:002014-07-11T05:48:08.128-07:00Thanks for this interesting post.
I have implemen...Thanks for this interesting post.<br /><br />I have implemented your model in BUGS and the mean results were n=106 (like your solution), p1=0.22, p2=0.17. However, the MLE estimate that I got is 77, which is a bit far from your 72 estimate. The prior for n must have been different (I used a truncated exponential with very low lambda) but even so it seems a big difference.<br /><br />The code and results are here: http://www.di.fc.ul.pt/~jpn/r/bugs/lincoln.html<br /><br />Cheers,João Netohttps://www.blogger.com/profile/05560718055133816500noreply@blogger.com