{"id":2893,"date":"2012-04-08T14:57:07","date_gmt":"2012-04-08T18:57:07","guid":{"rendered":"http:\/\/mathgradblog.williams.edu\/?p=2893"},"modified":"2012-04-08T14:57:07","modified_gmt":"2012-04-08T18:57:07","slug":"trust-numbers-much","status":"publish","type":"post","link":"https:\/\/blogs.ams.org\/mathgradblog\/2012\/04\/08\/trust-numbers-much\/","title":{"rendered":"Should we Trust Numbers that Much?"},"content":{"rendered":"
\"\"<\/a>

Taken from http:\/\/tinyurl.com\/cv2dcxk<\/p><\/div>\n

Nowadays, if one reads an article from a newspaper or a magazine, it is very likely there will be mention of some statistics, whether about the percentage of Americans that favor a candidate, of students that claim to be satisfied with their investment in getting a college degree, or of many more things\u00a0about which\u00a0someone may be puzzled to know some study (or research)\u00a0has been published. For example, take a look at this paragraph from \u201cIs Mitt Romney Damaged Goods after a Brutal Primary Season<\/a>.\u201d<\/p>\n

<\/em>Romney\u2019s negatives reached a record high last month with 50 percent of all voters, and 52 percent of registered voters, in a Washington\u00a0\u00a0\u00a0 Post\/ABC News poll that reflected a\u00a0dim view\u00a0of him as a prospective president. A Gallup \u201cSwing States\u201d survey shows President Obama leading Romney 51 to 42 percent, and among women 52 to 35 percent\u2014a huge gender gap. (The Daily Beast)<\/em><\/p>\n

 <\/p>\n

<\/p><\/blockquote>\n

What about this one quoted in Josh Fishman\u2019s article<\/a> in the Chronicle of Higher Education:<\/em><\/p>\n

<\/em>More than 60 percent of college undergraduates, and more than 40 percent of graduate students, admit to cheating in some way on their written work, according to a national survey<\/a> by Clemson University\u2019s International Center for Academic Integrity.<\/em><\/p><\/blockquote>\n

Here is another rather long example<\/a> from Psychology Today<\/em>, which may scare or impress someone with a modest math training:<\/em><\/p>\n

Half of the respondents simply estimated the probability that Jack (Dick) was an engineer. This part of the experiment was set up to replicate Kahneman & Tversky\u2019s (1973) original finding of base rate neglect. The replication was only partially successful. Jack was judged to be more likely to be an engineer when the base rate probability of being an engineer was high (M = 77%) than when it was low (M = 56%, t(58) = 2.25, p = .03. However, Dick was not judged to be significantly more likely to be an engineer given a high (M = 53%) instead of a low (M = 44%) base rate, t(58) = 1.64. Note that the statistical demonstration of complete base rate neglect in this paradigm requires a failure to reject a null hypothesis. Our data failed to replicate this failure in the case of Jack. Some reviewers of this field of research have concluded that base rate neglect is rarely as complete as it was in Kahneman & Tversky\u2019s original demonstration (Koehler, 1996).<\/em><\/p>\n

Given only partial base rate neglect, the novel idea of reducing this neglect with an anchoring manipulation had only limited room to show itself. To test this idea, we asked the other half of the participants to first respond to the question \u201cWould you say that the probability that Jack (Dick) is an engineer is greater than 70% (lower than 30%)? Virtually all responses were \u201cno.\u201d That was as it should be for most conditions because no rational or heuristic argument could be made for estimates lying<\/a> beyond the base rates. The exception to this rule is Jack, who, looking like an engineer, should (and did) receive an estimate greater than 70% when the base rate was 70%. In this exceptional case, anchoring had no effect. Jack was judged to be an engineer with an average chance of 77 and 78%, respectively with and without the anchor. Conversely, when participants first rejected the anchor of 30% as implausible they judged Jack to be less likely to be an engineer (M = 56%) than when they did not respond to the anchor first (M = 66%), t(58) = 4.75. This is good support for the anchoring-can-reduce-base-rate-neglect hypothesis. For judgments of Jack, the effect size of the difference more than doubled from the replication to the anchoring condition, d = .58 to 1.25. Anchoring also improved judgments of Dick. Without anchors, there was base rate neglect, as indicated by a nonsignificant difference between the two estimates. With anchors, the difference between (M = 54%) and (M = 39%) was significant, t(58) = 3.52, p = .001). The effect size doubled from d = .43 to .91.<\/em><\/p><\/blockquote>\n

One then may ask what explains this trend? It seems the assumption behind the overuse of numbers is that they are taken to be \u2018objective\u2019; therefore, they should transcend any bias from the authors, which then implies whatever they assert with those numbers must be true. Another implication may be that no one wants to take the responsibility of any error in one\u2019s research since everybody can blame it\u00a0on the numbers.<\/p>\n

This is a delicate if not dangerous situation: many of these arguments are supposed to be addressed to an audience not necessarily trained in advanced mathematics to evaluate those claims made from these numbers; unfortunately, the people who\u00a0could be able to evaluate some of these claims are a minority. This leads one to believe that major decisions may be taken on false assumptions, whether intentional or not,\u00a0or from just plain misinformation.<\/p>\n

In this context, should the use of mathematics be more controlled? Should the presence of mathematicians be more obvious in places where quantitative methods are used to guarantee their validity and justly assess their limitations? Would that be considered as an invasion by mathematicians? Even better, would this apparent misuse of mathematics be a motivation to teach students mathematics in such a way that they can become better judge of these numbers rather than teaching them\u00a0rudimentary rules they hardly understand let alone be able to use later for themselves?<\/p>\n

 <\/p>\n

<\/div>","protected":false},"excerpt":{"rendered":"

Nowadays, if one reads an article from a newspaper or a magazine, it is very likely there will be mention of some statistics, whether about the percentage of Americans that favor a candidate, of students that claim to be satisfied … Continue reading →<\/span><\/a><\/p>\n

<\/div>\n","protected":false},"author":33,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[8,14,15,20],"tags":[],"class_list":["post-2893","post","type-post","status-publish","format-standard","hentry","category-general","category-math-in-pop-culture","category-mathematics-in-society","category-teaching"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p3gbww-KF","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blogs.ams.org\/mathgradblog\/wp-json\/wp\/v2\/posts\/2893","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.ams.org\/mathgradblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.ams.org\/mathgradblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.ams.org\/mathgradblog\/wp-json\/wp\/v2\/users\/33"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.ams.org\/mathgradblog\/wp-json\/wp\/v2\/comments?post=2893"}],"version-history":[{"count":0,"href":"https:\/\/blogs.ams.org\/mathgradblog\/wp-json\/wp\/v2\/posts\/2893\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.ams.org\/mathgradblog\/wp-json\/wp\/v2\/media?parent=2893"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.ams.org\/mathgradblog\/wp-json\/wp\/v2\/categories?post=2893"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.ams.org\/mathgradblog\/wp-json\/wp\/v2\/tags?post=2893"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}