{"id":3229,"date":"2020-08-04T14:04:13","date_gmt":"2020-08-04T18:04:13","guid":{"rendered":"http:\/\/blogs.ams.org\/matheducation\/?p=3229"},"modified":"2020-08-18T09:17:13","modified_gmt":"2020-08-18T13:17:13","slug":"3229","status":"publish","type":"post","link":"https:\/\/blogs.ams.org\/matheducation\/2020\/08\/04\/3229\/","title":{"rendered":"Writing Good Questions for the Internet Era"},"content":{"rendered":"

Jeff Suzuki<\/p>\n

CUNY Brooklyn<\/p>\n

The forced conversion to distance learning in Spring 2020 caught most of us off-guard. One of the biggest problems we face is the existence of free or freemium online calculators that show all steps required to produce a textbook perfect solution.\u00a0 A student can simply type in “Solve ” or “Find the derivative of ” or “Evaluate ” or “Solve ,” and the site will produce a step-by-step solution indistinguishable from one we’d show in class.\u00a0 With Fall 2020 rapidly approaching, and no sign that distance learning will be abandoned, we must confront a painful reality:\u00a0\u00a0 Every question that can be answered by following a sequence of steps is now meaningless as a way to measure student learning.<\/p>\n

So how can we evaluate student learning?\u00a0 Those of us fortunate enough to teach courses with small enrollments have a multitude of options:\u00a0 oral exams; semester-long projects; student interviews.\u00a0 But for the rest of us, our best option is to ask “internet resistant” questions.\u00a0\u00a0\u00a0 Here are three strategies for writing such questions:<\/p>\n

\u25cf\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Require inefficiency.<\/p>\n

\u25cf\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Limit the information.<\/p>\n

\u25cf\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Move the lines<\/p>\n

<\/p>\n

Require Inefficiency<\/strong><\/p>\n

One of the goals of mathematics education is developing adaptive expertise: the ability to identify which of the many possible algorithms is the best to use on a particular problem.<\/p>\n

For example, consider a quadratic equation.\u00a0 We have at least two ways of solving quadratic equations:\u00a0 by factoring; or by the quadratic formula.\u00a0 Which do we use?\u00a0 Since the quadratic formula always works, there’s no obvious reason why we would ever want to use anything else.\u00a0 But sometimes using the quadratic formula is like using a chainsaw to cut a dinner roll:\u00a0 we wouldn’t use it on “Solve $(3x-7)(2x+5) = 0$ ,” and we probably wouldn’t use it on “Solve $x^2-9 = 0$,” though we’d almost certainly use it on “Solve $6x^2 – 19x – 36 = 0$.”\u00a0\u00a0 The boundary between the problems we’d attempt to solve by factoring and the problems we’d solve using the quadratic formula can’t be taught:\u00a0 every student has to find it for themselves through firsthand experience.<\/p>\n

It should be clear that requiring inefficiency is a possibility every time there is more than one way to solve a problem.\u00a0 This approach works even better when one method is clearly (to us) less efficient.\u00a0\u00a0 Indeed, the least efficient method is one that doesn’t work, and in some ways, requiring inefficiency in such cases may give us more insight into student learning than their ability to solve a problem.<\/p>\n

For example, consider the problem:<\/p>\n

If possible, solve by factoring:\u00a0 \u00a0$x^2 \u2013 3x \u2013 12 = 0$.\u00a0 If not possible, show why; then solve using the quadratic formula.<\/p>\n

Since the quadratic expression is irreducible over the integers, no online calculator will produce a factorization.\u00a0 Thus, a student can’t simply look up the answer.\u00a0 More importantly, in order to provide an answer, they must check every possible pair of factors (and show that none of them work).<\/p>\n

There’s an added bonus.\u00a0 On the same exam, we might ask students to factor various quadratic expressions. We argue that a student’s attempt to factor $x^2-3x-12$ \u00a0will actually reveal more about whether a student understands factoring than the successful factorization of an expression like $6x^2 + 19x \u2013 36$.\u00a0 Thus, we can omit straight factorization questions (which, in any case, can be “solved” by an online calculator).<\/p>\n

Limit the Information<\/strong><\/p>\n

Another way to thwart the use of internet calculators is to provide incomplete data.\u00a0 For example, Wolfram Alpha can find the derivative of any function—provided you give it the function.\u00a0 Thus we might ask students to solve problems without giving them equations.<\/p>\n

This might sound hard to do, but it’s actually pretty easy.\u00a0 Since the 1990s, state and national mathematics standards have called for increased use of graphical and tabular representations, so source material is plentiful.\u00a0 Even the most traditional texts include problems based on interpreting graphical and tabular data.\u00a0\u00a0 For example:<\/p>\n

Suppose you know $f(3) = 5$ \u00a0and $f'(3) = -4$ .\u00a0 Let $h(x) = ln f(x)$ .\u00a0 Find\u00a0 $h'(3)$.<\/p>\n

While this is an algorithmic question that can be easily answered by invoking the chain rule, doing so relies on correctly interpreting the written statements about the function and derivative values.\u00a0 As such, it is currently beyond the capability of online calculators.<\/p>\n

We can also present data graphically:<\/p>\n

The graphs of y = f(x) (solid) and y = g(x) (dashed) are shown:<\/p>\n

\"\"<\/a><\/p>\n

Find the sign of $(fg)’)(0)$.<\/p>\n

Again, this is an algorithmic question that can be answered by invoking the product rule.\u00a0 However, it relies on being able to extract information from a graph, then make a quantitative argument based on the signs of the functions and their derivatives.<\/p>\n

Moving the Lines<\/strong><\/p>\n

Requiring inefficiency and limiting information should be viewed as stopgap measures at best.\u00a0\u00a0 Thus, when calculators were first introduced, math teachers insisted on “exact answers,” since the student who returned the answer “1.4142135” instead of $\\sqrt{2}$ was clearly using a calculator.\u00a0 But now, even a \\$10 calculator can return “exact answers” like $\\frac {3+\\sqrt {5}} 2$ , so this distinction is no longer useful as a way of distinguishing between students who used a calculator and students who didn’t.\u00a0 Similarly, while I’m not aware of any app that allows for the user to select a solution method, or that can read graphical or tabular data, there’s no a priori reason why there couldn’t be one.\u00a0 This means we need a more powerful method of creating internet resistant questions that can adapt to advances in technology.\u00a0 This leads to a strategy I call “moving the lines.”<\/p>\n

To begin with, it’s important to understand that the problem “Solve\u00a0 $x^2 \u2013 3x \u2013 12 = 0$” does not exist outside of a mathematics classroom.\u00a0 So we should ask two questions:<\/p>\n

\u25cf\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Where did this problem come from?\u00a0 This moves the “starting line,” where the problem begins.<\/p>\n

\u25cf\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Why do we want the solution?\u00a0\u00a0 This moves the “finish line,” where the problem ends.<\/p>\n

Our long-term goal as mathematics educators should be to shift the lines and turn a sprint into a marathon.<\/p>\n

Let’s consider this problem.\u00a0 What leads to\u00a0 “Solve $x^2 \u2013 3x \u2013 12 = 0$?”\u00a0 For that, we might consider some of the basic steps in solving any quadratic equation.\u00a0\u00a0\u00a0 One of those steps is to get the equation into standard form. So our problem “Solve\u00a0 $x^2-3x-12 = 0$” might have come from “Solve $x^2 \u2013 3x = 12$.”\u00a0 In fact, you’ve probably asked this question before, specifically to identify students who failed to understand the necessity of getting the equation into standard form.<\/p>\n

Now where might we have gotten a problem like that?\u00a0 We might have gotten it from “Solve $x(x-3) = 12$.”\u00a0\u00a0 In fact, you’ve probably asked this type of question as well, to identify the students who failed to understand the zero product property.<\/p>\n

Note that we still have an equation that can be dropped into an online calculator, so the next step is important:\u00a0 What type of question leads to a product equal to a number?\u00a0 There are many times we multiply two numbers to get a quantity of interest; for example, the product of a rectangle’s length and width gives us the area.\u00a0 This takes us to the problem:<\/p>\n

A rectangle has an area of twelve square feet, and its width is three feet less than its length.\u00a0 Find the length of the rectangle.<\/em><\/p>\n

In order to answer this question, a student would have to translate the given information into a mathematical form.\u00a0\u00a0 This is beyond the capability of online calculators (especially if, as in this case, the numbers are also spelled out).\u00a0 If you enter the question into Google, you’ll get examples of similar problems, but no solution, effectively reducing you to your class notes and textbook.\u00a0 If you’re clever enough to switch numbers for the words, you’ll get an answer—which is incorrect (4 feet).<\/p>\n

We can further improve the problem by changing the finish line.\u00a0 Remember that once a student translates this problem into the equation $x(x-3) = 12$, an online calculator can produce the algebraic solution, showing all the steps.\u00a0 One way to further blunt the ability of the online calculator to answer all questions is to require another step beyond the mathematical solution.\u00a0 Thus we should ask why we’d want the answer.<\/p>\n

Let’s consider:\u00a0 we obtain the length (and width, since we know it’s three feet less than the length).\u00a0 So why would we want the length and width of a rectangle?\u00a0 There are three obvious possibilities:\u00a0 to find the rectangle’s area; to find the rectangle’s perimeter; and to find the rectangle’s diagonal.\u00a0 Since we already know the area, we might want either the perimeter or the diagonal.\u00a0 So we could ask:<\/p>\n

A rectangle has an area of twelve square feet, and its width is three feet less than its length.\u00a0 Find the perimeter of the rectangle.<\/em><\/p>\n

Even better:<\/p>\n

A homeowner wants to fence a garden in the shape of a rectangle.\u00a0 The garden must have an area of twelve square feet, where the width is three feet less than its length.\u00a0 The fence will cost two dollars per foot.\u00a0 How much will it cost to enclose the garden?<\/em><\/p>\n

The best part about this approach is that as technology advances, we can shift the lines in response.\u00a0 Perhaps some day we’ll be able to enter the above problem into a search engine and get the correct answer.\u00a0 So the next step will be to shift the lines again:\u00a0 move the starting point further back by imagining where the problem might come from; and move the finish line further forward by considering why we’d want to know the cost.<\/p>\n

The Road Ahead<\/strong><\/p>\n

Notice that we end with something that might be called a “real world” problem.\u00a0 But a homeowner rarely has to build a garden with a specific area and relationship between the sides:\u00a0 it would be a stretch to call the problem above a real world example of how to use mathematics.<\/p>\n

What’s more important is that real world problems don’t come with instructions on how to solve them, so they must be solved inefficiently, by trying different approaches until we find one that works.\u00a0 Real world problems don’t come with formulas attached to them, so they must be solved without complete information.\u00a0 And real world problems often change, so we must expect that the starting and finishing lines will change on us.<\/p>\n

What this means is that regardless of when or if we can resume traditional resource-restricted exams, we should consider requiring inefficiency, limiting information, and shifting the lines on all our assessments.\u00a0 Sooner or later, our students will leave our classroom.\u00a0 If what they learned can be replaced by someone using a free internet app, then they can be replaced by a free internet app.\u00a0 So it’s not just about making our questions internet resistant:\u00a0 it’s also about making our students internet resistant.<\/p>\n

Addendum<\/strong><\/p>\n

We’re stronger together.\u00a0 Readers interested in sharing their “internet resistant” questions should email them to me at jsuzuki@brooklyn.cuny.edu, and I’ll put up a selection of these in a later post.<\/p>\n

<\/div>","protected":false},"excerpt":{"rendered":"

Jeff Suzuki CUNY Brooklyn The forced conversion to distance learning in Spring 2020 caught most of us off-guard. One of the biggest problems we face is the existence of free or freemium online calculators that show all steps required to … Continue reading →<\/span><\/a><\/p>\n

<\/div>\n","protected":false},"author":140,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[197,29,207,198,257,158],"tags":[144,31,132,110,319],"class_list":["post-3229","post","type-post","status-publish","format-standard","hentry","category-active-learning-in-mathematics-series-2015","category-communication","category-curriculum","category-k-12-education","category-mathematics-education-research","category-online-education","tag-curriculum","tag-education","tag-k-12-mathematics","tag-mathematics-education","tag-online-learning"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/s6C2AC-3229","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/posts\/3229","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/users\/140"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/comments?post=3229"}],"version-history":[{"count":16,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/posts\/3229\/revisions"}],"predecessor-version":[{"id":3310,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/posts\/3229\/revisions\/3310"}],"wp:attachment":[{"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/media?parent=3229"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/categories?post=3229"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/tags?post=3229"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}