Here is a test for you. Let\u2019s say 300 mathematicians were polled concerning how many hours of TV they watch per week. What does it mean to say that a 95% confidence interval for the average number of hours of television watched by a mathematician per week is the interval from 1 to 3 hours? Here are some reasonable sounding answers\u2026<\/p>\n
Whatever your answer to the question above, think about whether it is equivalent to the following correct answer: the PROCESS used to create the confidence interval has a 95% chance of success\u2014that is, there is a 95% probability that whatever interval is created through this process will contain the true average. While it is conceivable (but unlikely) that I could find enough mathematicians to replicate my experiment 100 times, I\u2019m still not sure what this tells me since I may get (possibly very) different upper and lower bounds for the confidence interval each time I perform the experiment.<\/p>\n
I probably sound kind of like a really annoying Sophomore by now, but here is my honest question: what is the most reasonable way to practically use confidence intervals? Along these lines, it seems that psychologists are strongly considering using alternative methods (to the currently accepted significance level) for reporting the results of their experiments. Under consideration is the reporting of confidence intervals, which do not rely on null hypothesis testing.<\/p>\n
I guess one question is \u2013 is this mainly a problem with education in that people don\u2019t know what a confidence interval is, or is it that the measurement itself is not serving the purpose that most people have come to use it for<\/p>\n
So hopefully you have some ideas for me, and maybe now someone will be inspired to conduct a survey on TV-watching habits of mathematicians at the next JMM\u2019s.<\/p>\n
These reflections are all inspired by:<\/p>\n
1) Alex Etz, a UT graduate student at\u00a0The Etz-Files: Blogging About Science, Statistics, and Brains — Nov. 16th and Nov. 20th posts entitled Can Confidence Intervals Save Pyschology?\u00a0<\/em>http:\/\/nicebrain.wordpress.com\/2014\/11\/16\/can-confidence-intervals-save-psychology-part-1\/<\/a><\/p>\n 2) From my friend Suz Ward at AIR — July post entitled Confident or Credible? Two Metrics of Uncertainty for Decision Making<\/em> \u00a0http:\/\/www.air-worldwide.com\/Blog\/Confident-or-Credible–Two-Metrics-of-Uncertainty-for-Decision-Making\/<\/a><\/p>\n 3) Christian Jarrett at the BPS Research Digest– Nov. 14th post entitled\u00a0Reformers say psychologists should change how they report their results, but does anyone understand the alternative?<\/em>\u00a0http:\/\/digest.bps.org.uk\/2012\/08\/phew-made-it-how-uncanny-proportion-of.html<\/a><\/p>\n