{"id":951,"date":"2015-09-20T00:01:04","date_gmt":"2015-09-20T04:01:04","guid":{"rendered":"http:\/\/blogs.ams.org\/matheducation\/?p=951"},"modified":"2015-12-28T07:17:04","modified_gmt":"2015-12-28T12:17:04","slug":"active-learning-in-mathematics-part-ii-levels-of-cognitive-demand","status":"publish","type":"post","link":"https:\/\/blogs.ams.org\/matheducation\/2015\/09\/20\/active-learning-in-mathematics-part-ii-levels-of-cognitive-demand\/","title":{"rendered":"Active Learning in Mathematics, Part II: Levels of Cognitive Demand"},"content":{"rendered":"<p><i><span style=\"font-weight: 400\">By Benjamin Braun, <a href=\"http:\/\/blogs.ams.org\/matheducation\/about-the-editors\" target=\"_blank\">Editor-in-Chief<\/a>, University of Kentucky; Priscilla Bremser, <a href=\"http:\/\/blogs.ams.org\/matheducation\/about-the-editors\" target=\"_blank\">Contributing Editor<\/a>, Middlebury College; Art Duval, <a href=\"http:\/\/blogs.ams.org\/matheducation\/about-the-editors\" target=\"_blank\">Contributing Editor<\/a>, University of Texas at El Paso; Elise Lockwood, <a href=\"http:\/\/blogs.ams.org\/matheducation\/about-the-editors\" target=\"_blank\">Contributing Editor<\/a>, Oregon State University; and Diana White, <a href=\"http:\/\/blogs.ams.org\/matheducation\/about-the-editors\" target=\"_blank\">Contributing Editor<\/a>, University of Colorado Denver.<\/span><\/i><\/p>\n<p><i><span style=\"font-weight: 400\">Editor\u2019s note: This is the second article in a series devoted to active learning in mathematics courses. \u00a0The other articles in the series can be found <a href=\"http:\/\/blogs.ams.org\/matheducation\/tag\/active-learning-series-2015\/\" target=\"_blank\">here<\/a>.<\/span><\/i><\/p>\n<p>Mathematics faculty are well-aware that students face challenges when encountering difficult problems, and it is common to hear instructors remark that successful students have high levels of \u201cmathematical maturity,\u201d or are particularly \u201ccreative,\u201d or write \u201celegant\u201d solutions to problems. \u00a0To appreciate research results regarding active learning, it is useful to make these ideas more precise. \u00a0Motivated by research in education, psychology, and sociology, language has been developed that can help mathematicians clarify what we mean when we talk about difficulty levels of problems, and the types of difficulty levels problems can have. This expanded vocabulary is in large part motivated by&#8230;<\/p>\n<blockquote><p><span style=\"font-weight: 400\">&#8230;the \u201ccognitive revolution\u201d [of the 1970\u2019s and 1980\u2019s]&#8230; [which] produced a significant reconceptualization of what it means to understand subject matter in different domains. There was a fundamental shift from an exclusive emphasis on knowledge &#8212; what does the student know? &#8212; to a focus on what students know and can do with their knowledge. The idea was not that knowledge is unimportant. Clearly, the more one knows, the greater the potential for that knowledge to be used. Rather, the idea was that having the knowledge was not enough; being able to use it in the appropriate circumstances is an essential component of proficiency.<\/span><\/p>\n<p><span style=\"font-weight: 400\">&#8212; Alan Schoenfeld, <\/span><i><span style=\"font-weight: 400\">Assessing Mathematical Proficiency<\/span><\/i><span style=\"font-weight: 400\"> [17]<\/span><\/p><\/blockquote>\n<p><span style=\"font-weight: 400\">In this article, we will explore the concept and language of &#8220;level of cognitive demand&#8221; for tasks that students encounter. \u00a0A primary motivation for our discussion is the important observation in the 2014 Proceedings of the National Academy of Science (PNAS) article \u201c<\/span><a href=\"http:\/\/www.pnas.org\/content\/111\/23\/8410.abstract\" target=\"_blank\"><span style=\"font-weight: 400\">Active learning increases student performance in science, engineering, and mathematics<\/span><\/a><span style=\"font-weight: 400\">\u201d by Freeman, et al. [8], that active learning has a greater impact on student performance on concept inventories than on instructor-written examinations. \u00a0Concept inventories are \u201ctests of the most basic conceptual comprehension of foundations of a subject and not of computation skill\u201d and are \u201cquite different from final exams and make no pretense of testing everything in a course\u201d [5]. \u00a0The <\/span><a href=\"http:\/\/www.ams.org\/notices\/201308\/rnoti-p1018.pdf\" target=\"_blank\"><span style=\"font-weight: 400\">Calculus Concept Inventory<\/span><\/a><span style=\"font-weight: 400\"> is the most well-known inventory in mathematics, though compared to disciplines such as physics these inventories are less robust since they are in relatively early stages of development. \u00a0Freeman et al. state:<\/span><\/p>\n<blockquote><p><span style=\"font-weight: 400\">Although student achievement was higher under active learning for both [instructor-written course examinations and concept inventories], we hypothesize that the difference in gains for examinations versus concept inventories may be due to the two types of assessments testing qualitatively different cognitive skills. \u00a0This is consistent with previous research indicating that active learning has a greater impact on student mastery of higher- versus lower-level cognitive skills&#8230;<\/span><\/p><\/blockquote>\n<p>After introducing levels of cognitive demand in this article, our next article in this series will directly connect this topic to active learning techniques that are frequently used and promoted for postsecondary mathematics courses.<\/p>\n<p><!--more--><\/p>\n<p><b>Bloom\u2019s Taxonomy and its Variants<\/b><\/p>\n<p><span style=\"font-weight: 400\">A well-known and long-established framework in educational psychology is Bloom&#8217;s taxonomy [2]. \u00a0In 1956, Benjamin Bloom and a team of educational psychologists outlined multiple levels of skills in the cognitive domain of learning, increasing from simple to complex. \u00a0These are often simplified into six skill levels: knowledge, comprehension, application, analysis, synthesis, evaluation. \u00a0By associating verbal cue words with each level, they categorized test questions over a variety of topics at the college level, and found that <\/span><i><span style=\"font-weight: 400\">over 95%<\/span><\/i><span style=\"font-weight: 400\"> of these questions were at the very lowest level, \u201crecall of knowledge\u201d [11, p. 1]. \u00a0Since these original findings, which were further developed in a second volume published in 1964, the core ideas of Bloom\u2019s taxonomy have been widely used in education across disciplines.<\/span><\/p>\n<p><span style=\"font-weight: 400\">The original taxonomy has been extended and adapted by many researchers in educational psychology. \u00a0For example, Anderson et al. [1] developed a two-dimensional extension of Bloom\u2019s taxonomy with a cognitive process dimension (remember, understand, apply, analyze, evaluate, create) similar to Bloom&#8217;s taxonomy, but also with a knowledge dimension (factual knowledge, conceptual knowledge, procedural knowledge, and metacognitive knowledge) &#8212; a taxonomy table encoding this appears below. \u00a0When categorizing a task by this taxonomy, the cognitive process is represented by the verb used when specifying the task (what the student is doing) and the knowledge process dimension corresponds to the noun (what kind of knowledge the student is working with). \u00a0In 2002, a <\/span><a href=\"http:\/\/www.tandfonline.com\/toc\/htip20\/41\/4\" target=\"_blank\"><span style=\"font-weight: 400\">special volume of the journal <\/span><i><span style=\"font-weight: 400\">Theory Into Practice<\/span><\/i><\/a><span style=\"font-weight: 400\"> was devoted to this revised taxonomy; examples of applications of this taxonomy can be found throughout the volume.<\/span><\/p>\n<p>&nbsp;<\/p>\n<table style=\"border: 1px solid black;border-collapse: collapse\">\n<tbody>\n<tr>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Remember<\/span><\/td>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Understand<\/span><\/td>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Apply<\/span><\/td>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Analyze<\/span><\/td>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Evaluate<\/span><\/td>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Create<\/span><\/td>\n<\/tr>\n<tr>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Factual Knowledge<\/span><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<\/tr>\n<tr>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Conceptual Knowledge<\/span><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<\/tr>\n<tr>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Procedural Knowledge<\/span><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<\/tr>\n<tr>\n<td style=\"border: 1px solid black\"><span style=\"font-weight: 400\">Metacognitive Knowledge<\/span><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<td style=\"border: 1px solid black\"><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400\">An important shortcoming of each of these taxonomies for mathematicians is that the specific descriptors used for the different levels aren&#8217;t always appropriate for mathematics. \u00a0For instance, in Bloom&#8217;s taxonomy, application comes after comprehension, which does make sense in a general context. \u00a0But trying to apply this to mathematics, it is too easy to put routine word problems in the \u201capplication\u201d category. \u00a0The idea of &#8220;application&#8221; in the general sense is to take ideas presented in one context and be able to use them in a somewhat new setting, but in mathematics the word \u201capplication\u201d can be used to represent both the development of a mathematical model to fit a situation or data set and the \u201ccookbook\u201d application of a previously-established mathematical model; most word problems in textbooks fit into the latter category.<\/span><\/p>\n<p><b>Specialized Cognitive Taxonomies and General Student Intellectual Development<\/b><\/p>\n<p>Around the same time as [1], several papers appeared that used taxonomies specialized to mathematics, e.g., [15, 19, 20, 21]. \u00a0These have the two-dimensional nature of [1], with the columns or verbs replaced by labels that are specific to mathematics, while the rows or nouns simply correspond to different topics in mathematics. \u00a0In 2006, Andrew Porter [14] explained it this way:<\/p>\n<blockquote><p><span style=\"font-weight: 400\">Unfortunately, defining content in terms of topics has proven to be insufficient at least if explaining variance in student achievement is the goal [9]. For example, knowing whether or not a teacher has taught linear equations, while providing some useful information, is insufficient. What about linear equations was taught? Were students taught to distinguish a linear equation from a non-linear equation? Were students taught that a linear equation represents a unique line in a two space and how to graph the line? For every topic, content can further be defined according to categories of cognitive demand. In mathematics cognitive demand might distinguish memorize; perform procedures; communicate understanding; solve non-routine problems; conjecture, generalize, prove.<\/span><\/p><\/blockquote>\n<p>More details about this taxonomy of levels of cognitive demand can be found in [15]. \u00a0A comparison of various such taxonomies can be found in [15].<\/p>\n<p>Similarly, several papers of Mary Kay Stein and various co-authors [19, 20, 21] analyze mathematical tasks and how they are implemented, focusing on middle school, using four levels of cognitive demand: Memorization; procedures without connections; procedures with connections; and \u201cdoing mathematics\u201d. \u00a0They identify the first two levels as \u201clow-level\u201d, matching the first two levels of [15]; and they identify the last two levels as \u201chigh-level\u201d, matching the last three levels of [15].<\/p>\n<p>There are also broad models for student intellectual development across not only individual topics but their entire college experience. \u00a0One of the first such models is due to William Perry, and it can be (overly) simplified into the following description. \u00a0Most college students will begin with the belief that there are right and wrong answers to questions, and that professors hold the knowledge of which these are. \u00a0As students progress through their studies, they realize that sometimes their teachers are not always aware of the answers to questions, and also that answers can be more subtle than merely \u201cright\u201d or \u201cwrong.\u201d \u00a0After this realization, students often enter a phase of relativism, where everyone\u2019s opinions are equally valid. \u00a0In the final stages of intellectual development, students recognize that different areas of intellectual inquiry have different standards and (some students) develop a balance between intellectual independence and commitment to the discipline. \u00a0The Perry model has been refined and revised by many psychologists to account for diverse student experiences with respect to gender and other factors; an excellent survey of these developments, with pedagogical implications, has been given by Felder and Brent [6,7].<\/p>\n<p>As Thomas Rishel points out [16], students in the early stages of the Perry model or one of its variants often enjoy mathematics precisely because all the answers are perceived as known, and they frequently value mathematical problems that focus on verification of these truths. \u00a0As these students begin to encounter complicated modeling problems, or as they are first asked to seriously participate in proof-based mathematical reasoning, the cognitive load of such tasks can be much higher than for students who have developed further along this model. \u00a0Thus, the intellectual stage of development for a given student can impact the level of cognitive demand for various tasks and problems they will encounter in mathematics courses.<\/p>\n<p><b>Practical Issues: Level Identification and Task Assessment<\/b><\/p>\n<p><span style=\"font-weight: 400\">Given these theoretical frameworks for both cognitive engagement and intellectual development, a practical challenge for instructors is to use these frameworks effectively to increase the quality of teaching and learning in the classroom.<\/span><b> \u00a0<\/b><span style=\"font-weight: 400\">With any of the cognitive taxonomies, it can be hard to assess precisely which level(s) a given student task is hitting. \u00a0The taxonomy tables discussed in previous sections provide instructors with tools to produce reasonable cognitive demand analysis of the tasks they give students. \u00a0Engagement with all cognitive levels is necessary for deep learning to take place, so it is important that mathematics faculty identify and provide students with tasks representing a range of levels. \u00a0Since lower-level tasks are typically already most prevalent, and easiest to assess both in terms of time and resources, faculty have to make the effort to bring in the higher levels. \u00a0As a result, three challenges for instructors are to identify high-quality mathematical activities for students at higher levels of cognitive demand, to develop methods for assessing student work on such activities, and to create or make use of institutional programs, culture, and resources to support the use of high-quality activities. \u00a0We will comment on the third issue in our next article in this series.<\/span><\/p>\n<p><span style=\"font-weight: 400\">Some mathematics problems afford a wide range of cognitive engagement. \u00a0For example, in the K-12 setting Jo Boaler and others have promoted activities described as \u201clow-floor, high-ceiling\u201d (LFHC) [23]. \u00a0These are activities that can give students practice in lower levels of cognitive demand, but also are open-ended enough to eventually lead to (grade-appropriate) mathematical investigations with high-cognitive demand. \u00a0Good examples of problems that students can engage with all the way from elementary school procedures to the highest levels of cognitive demand, leading to college-level abstract topics, can be found on the <\/span><a href=\"https:\/\/www.youcubed.org\/task\/sums-investigation\/\" target=\"_blank\"><span style=\"font-weight: 400\">youcubed website<\/span><\/a><span style=\"font-weight: 400\">, on sites for <\/span><a href=\"https:\/\/www.mathcircles.org\/\" target=\"_blank\"><span style=\"font-weight: 400\">Math Circles<\/span><\/a><span style=\"font-weight: 400\">, and on <a href=\"https:\/\/www.yumpu.com\/en\/document\/view\/35667457\/math-teachers-circles-impacting-teachers-mathematical-\/3\" target=\"_blank\">sites<\/a> for <\/span><a href=\"http:\/\/www.mathteacherscircle.org\/\" target=\"_blank\"><span style=\"font-weight: 400\">Math Teachers\u2019 Circles<\/span><\/a><span style=\"font-weight: 400\">. \u00a0When students are working on LFHC problems, they have flexibility in how they navigate through the problem. \u00a0Unless explicit guidance is given regarding how students should investigate a LFHC problem, it is possible for them to spend most of their time working inside a small range of cognitive demand. \u00a0Consequently, it is important for instructors to provide some pathways or scaffolding for students to use when first engaging with such problems. \u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Though they are not as common as they deserve to be, mathematicians have developed a wide range of techniques for assessing high-cognitive demand tasks, including written assignments, group work, projects, portfolios, presentations, and more [3, 4, 10, 12, 13]. \u00a0However, task-appropriate techniques for assessing a given high-cognitive demand task can be challenging to identify and put in practice. \u00a0It is important that the method for assessing specific tasks be selected in the context of overall course assessment. \u00a0Some mathematicians have been experimenting with grading schemes that more directly support high-cognitive demand assignments, such as <\/span><a href=\"http:\/\/rtalbert.org\/blog\/2015\/Specs-grading-report-part-1\/\" target=\"_blank\"><span style=\"font-weight: 400\">specifications grading<\/span><\/a><span style=\"font-weight: 400\"> and <\/span><a href=\"https:\/\/blogs.cofc.edu\/owensks\/2015\/04\/01\/sbg-docs\/\" target=\"_blank\"><span style=\"font-weight: 400\">standards-based grading<\/span><\/a><span style=\"font-weight: 400\">. \u00a0Unfortunately, the fact remains that there is much to be learned about the efficacy of different methods of assessment [17]. \u00a0<\/span><\/p>\n<p><b>Active Learning and Theories of Learning<\/b><\/p>\n<p>Implicit in our discussion has been an important point that should be made explicit: as Stein et al. state [19], \u201c&#8230;cognitive demands are analyzed as the cognitive processes in which students actually engage as they go about working on the task\u201d as opposed to what students witness others doing. \u00a0Thus, it is not possible to discuss the cognitive level of a mathematical proof itself, though proofs certainly vary in level of sophistication. \u00a0Rather, one focuses on the cognitive level of what a student is asked to do with the proof: memorize the proof verbatim? construct a concrete example illustrating the proof method? derive a similar result using the same technique? analyze the proof in order to identify the key steps? compare the proof to a different proof of the same result? \u00a0These tasks are all different from the perspective of cognitive demand, hence they are not interchangeable from the perspective of student learning, yet they exhibit superficial similarities and would each generally be considered valuable for students to complete. \u00a0It is worth remarking that the verbs used in describing each of the tasks are helpful indicators of level of cognitive demand, as the taxonomies suggest.<\/p>\n<p><span style=\"font-weight: 400\">This observation brings us back to active learning, which by definition has as a primary goal to engage students through explicit mathematical tasks in the classroom, in view of peers, instructors, and teaching assistants. \u00a0One major effect of active learning techniques is that the mathematical processes and practices of students, which are tightly interwoven with high-cognitive achievement, are brought into direct confluence with peer and instructor feedback. \u00a0Thus, active learning techniques complement the shift in emphasis described by Schoenfeld, from received knowledge to committed engagement, of\u00a0the primary goal of student learning. \u00a0Active learning techniques are also well-aligned with contemporary theories of learning, for example constructivism, behaviorism, sociocultural theory, and others [18, 22]. \u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">As one example of this alignment, constructivism is based on the idea that people construct their own understanding and knowledge through their experiences rather than through the passive transfer of knowledge from one individual to another. This is a prominent theory of learning among mathematics education researchers with many refinements, e.g. radical constructivism remains agnostic about whether there actually is any objective truth\/reality, while social constructivism views individual thought and social interaction as inseparable with no model for a socially isolated mind. \u00a0Generally, constructivism\u2019s emphasis on the actions of the learner reinforces the need to emphasize consideration of the cognitive demands placed on students.<\/span><\/p>\n<p><b>Conclusion<\/b><\/p>\n<p><span style=\"font-weight: 400\">As research regarding the teaching and learning of postsecondary mathematics and science matures and becomes more well-known, both inside the mathematics community and beyond, significant evidence is building that active learning techniques have a strong impact on student achievement on high-cognitive demand tasks. \u00a0We began this article with recognition that mathematicians are fully aware of students\u2019 difficulties with mathematical tasks at all levels, and the observation that the mathematical community has developed language such as \u201cmathematical maturity\u201d and \u201celegance\u201d which is often applied to distinguish successful from unsuccessful student work. \u00a0Our main purpose in writing this survey of concepts related to levels of cognitive demand is to introduce mathematicians to the rich and complex set of ideas that have been developed in an attempt to distinguish different types of student activities and actions related to learning. \u00a0Given the current evidence supporting the positive impact of active learning techniques, mathematics faculty will have an increased need for a refined language with which to discuss both the successes and failures of our students and the efficacy of the large variety of active learning techniques that are available. \u00a0In our next post in this series, we will discuss the most prominent of these active learning techniques and environments with an eye toward both institutional constraints (as discussed in <\/span><a href=\"http:\/\/blogs.ams.org\/matheducation\/2015\/09\/10\/active-learning-in-mathematics-part-i-the-challenge-of-defining-active-learning\/\" target=\"_blank\"><span style=\"font-weight: 400\">Part I of this series<\/span><\/a><span style=\"font-weight: 400\">) and student learning in the context of levels of cognitive demand.<\/span><\/p>\n<p><b>References<\/b><\/p>\n<p><span style=\"font-weight: 400\">[1] Anderson, L.W. (Ed.), Krathwohl, D.R. (Ed.), Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J., &amp; Wittrock, M.C. <\/span><i><span style=\"font-weight: 400\">A taxonomy for learning, teaching, and assessing: A revision of Bloom&#8217;s Taxonomy of Educational Objectives (Complete edition)<\/span><\/i><span style=\"font-weight: 400\">. New York: Longman. 2001<\/span><\/p>\n<p><span style=\"font-weight: 400\">[2] Bloom, Benjamin, et al., eds. <\/span><i><span style=\"font-weight: 400\">Taxonomy of Educational Objectives: the classification of educational goals. Handbook I: Cognitive domain.<\/span><\/i><span style=\"font-weight: 400\"> New York: Longmans, Green. 1956<\/span><\/p>\n<p><span style=\"font-weight: 400\">[3] Benjamin Braun. Personal, Expository, Critical, and Creative: Using Writing in Mathematics Courses. <em>PRIMUS<\/em>, 24 (6), 2014, 447-464. <\/span><\/p>\n<p><span style=\"font-weight: 400\">[4] A. Crannell, G. LaRose, and T. Ratliff. <em>Writing Projects for Mathematics Courses: Crushed Clowns, Cars, and Coffee to Go.<\/em>\u00a0Mathematical Association of America, 2004.<\/span><\/p>\n<p><span style=\"font-weight: 400\">[5] J. Epstein. 2013. The Calculus Concept Inventory\u2014Measurement of the Effect of Teaching Methodology in Mathematics. <em>Notices of the American Mathematical Society.<\/em> 60 (8), 1018\u20131026.<\/span><\/p>\n<p><span style=\"font-weight: 400\">[6] Felder, Richard M. and Brent, Rebecca. <\/span><a href=\"http:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/j.2168-9830.2004.tb00816.x\/abstract\" target=\"_blank\"><span style=\"font-weight: 400\">The Intellectual Development of Science and Engineering Students. Part 1: Models and Challenges,<\/span><\/a><span style=\"font-weight: 400\"><em> Journal of Engineering Education<\/em>, Volume 93, Issue 4, October 2004, 269\u2013277.<\/span><\/p>\n<p><span style=\"font-weight: 400\">[7] Felder, Richard M. and Brent, Rebecca. <\/span><a href=\"http:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/j.2168-9830.2004.tb00817.x\/abstract\" target=\"_blank\"><span style=\"font-weight: 400\">The Intellectual Development of Science and Engineering Students. Part 2: Teaching to Promote Growth,<\/span><\/a><span style=\"font-weight: 400\"><em> Journal of Engineering Education<\/em>, Volume 93, Issue 4, October 2004, 279\u2013291.<\/span><\/p>\n<p><span style=\"font-weight: 400\">[8] Scott Freeman, Sarah L. Eddy, Miles McDonough, Michelle K. Smith, Nnadozie Okoroafor, Hannah Jordt, and Mary Pat Wenderoth. Active learning increases student performance in science, engineering, and mathematics. <\/span><i><span style=\"font-weight: 400\">Proc. Natl. Acad. Sci. U.S.A. <\/span><\/i><span style=\"font-weight: 400\">2014, 111, (23) 8410-8415<\/span><\/p>\n<p><span style=\"font-weight: 400\">[9] Gamoran, A., Porter, A.C., Smithson, J., &amp; White, P.A. (1997, Winter). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. <\/span><i><span style=\"font-weight: 400\">Educational Evaluation and Policy Analysis, 19<\/span><\/i><span style=\"font-weight: 400\">(4), 325-338. <\/span><\/p>\n<p><span style=\"font-weight: 400\">[10] Bonnie Gold, Sandra Z. Keith, William A. Marion, (eds). \u00a0<em>Assessment Practices in Undergraduate Mathematics<\/em>. Mathematical Association of America Notes #49, 1999.<\/span><\/p>\n<p><span style=\"font-weight: 400\">[11] Karin K. Hess. Exploring Cognitive Demand in Instruction and Assessment. National Center for Assessment, Dover, NH 2008. <\/span><a href=\"http:\/\/www.nciea.org\/publications\/DOK_ApplyingWebb_KH08.pdf\" target=\"_blank\"><span style=\"font-weight: 400\">http:\/\/www.nciea.org\/publications\/DOK_ApplyingWebb_KH08.pdf<\/span><\/a><\/p>\n<p><span style=\"font-weight: 400\">[12] \u00a0Reva Kasman. Critique That! Analytic writing assignments in advanced mathematics courses. <em>PRIMUS<\/em> XVI (2006) 1\u201315.<\/span><\/p>\n<p><span style=\"font-weight: 400\">[13] John Meier and Thomas Rishel. <\/span><i><span style=\"font-weight: 400\">Writing in the Teaching and Learning of Mathematics<\/span><\/i><span style=\"font-weight: 400\">. MAA Note #48, 1998.<\/span><\/p>\n<p><span style=\"font-weight: 400\">[14] Porter, Andrew. Curriculum Assessment, <\/span><span style=\"font-weight: 400\">In J. L. Green, G. Camilli, &amp; P. B. Elmore (Eds.), <em>Complementary methods for research in \u00a0education (3rd edition)<\/em>. Washington, DC: American Educational Research Association, 2006. \u00a0<\/span><a href=\"http:\/\/www.andyporter.org\/sites\/andyporter.org\/files\/papers\/CurriculumAssessment.pdf\" target=\"_blank\"><span style=\"font-weight: 400\">http:\/\/www.andyporter.org\/sites\/andyporter.org\/files\/papers\/CurriculumAssessment.pdf<\/span><\/a><\/p>\n<p><span style=\"font-weight: 400\">[15] Andrew C. Porter and John L. Smithson. \u00a0Defining, Developing, and Using Curriculum Indicators. \u00a0CPRE Research Report Series RR-048, December 2001. \u00a0Consortium for Policy Research in Education University of Pennsylvania Graduate School of Education. \u00a0<\/span><a href=\"https:\/\/secure.wceruw.org\/seconline\/Reference\/rr48.pdf\" target=\"_blank\"><span style=\"font-weight: 400\">https:\/\/secure.wceruw.org\/seconline\/Reference\/rr48.pdf<\/span><\/a><\/p>\n<p><span style=\"font-weight: 400\">[16] Rishel, Thomas. <\/span><i><span style=\"font-weight: 400\">Teaching First: A Guide for New Mathematicians<\/span><\/i><span style=\"font-weight: 400\">. MAA Notes #54, 2000<\/span><\/p>\n<p><span style=\"font-weight: 400\">[17] Schoenfeld, Alan H., ed. <\/span><i><span style=\"font-weight: 400\">Assessing Mathematical Proficiency<\/span><\/i><span style=\"font-weight: 400\">. MSRI Book Series, Volume 53, 2007.<\/span><\/p>\n<p><span style=\"font-weight: 400\">[18] B. Sriraman, &amp; L. English (Eds.). <\/span><i><span style=\"font-weight: 400\">Theories of mathematics education.<\/span><\/i><span style=\"font-weight: 400\"> New York: Springer, 2010.<\/span><\/p>\n<p>[19] Mary Kay Stein, Barbara W. Grover, and Marjorie Henningsen. \u00a0Building Student Capacity for Mathematical Thinking and Reasoning: An Analysis of Mathematical Tasks Used in Reform Classrooms. \u00a0<em>American Educational Research Journal<\/em>, Vol. 33, No. 2 (Summer, 1996), pp. 455-488<\/p>\n<p><span style=\"font-weight: 400\">[20] Stein, Mary Kay and Smith, Margaret Schwan. \u00a0\u201cMathematical Tasks as a Framework for Reflection: From Research to Practice.\u201d <\/span><i><span style=\"font-weight: 400\">Mathematics Teaching in the Middle School,<\/span><\/i><span style=\"font-weight: 400\"> Vol. 3, No. 4 (January 1998), pp. 268-275<\/span><\/p>\n<p><span style=\"font-weight: 400\">[21] Stein, Mary Kay and Smith, Margaret Schwan. \u201cReflections on Practice: Selecting and Creating mathematical Tasks: From Research to Practice.\u201d <\/span><i><span style=\"font-weight: 400\">Mathematics Teaching in the Middle School<\/span><\/i><span style=\"font-weight: 400\">, Vol. 3, No. 5 (February 1998), pp. 344- 350<\/span><\/p>\n<p><span style=\"font-weight: 400\">[22] T. Rowland &amp; P. Andrews (Eds.). <\/span><i><span style=\"font-weight: 400\">Master class in mathematics education: International perspectives on teaching and learning<\/span><\/i><span style=\"font-weight: 400\">. London: Continuum Publishers, 2014.<\/span><\/p>\n<p><span style=\"font-weight: 400\">[23] <\/span><a href=\"https:\/\/www.youcubed.org\/grade\/low-floor-high-ceiling\/\" target=\"_blank\"><span style=\"font-weight: 400\">https:\/\/www.youcubed.org\/grade\/low-floor-high-ceiling\/<\/span><\/a><\/p>\n<div style=\"margin-top: 0px; margin-bottom: 0px;\" class=\"sharethis-inline-share-buttons\" ><\/div>","protected":false},"excerpt":{"rendered":"<p>By Benjamin Braun, Editor-in-Chief, University of Kentucky; Priscilla Bremser, Contributing Editor, Middlebury College; Art Duval, Contributing Editor, University of Texas at El Paso; Elise Lockwood, Contributing Editor, Oregon State University; and Diana White, Contributing Editor, University of Colorado Denver. Editor\u2019s &hellip; <a href=\"https:\/\/blogs.ams.org\/matheducation\/2015\/09\/20\/active-learning-in-mathematics-part-ii-levels-of-cognitive-demand\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n<div style=\"margin-top: 0px; margin-bottom: 0px;\" class=\"sharethis-inline-share-buttons\" data-url=https:\/\/blogs.ams.org\/matheducation\/2015\/09\/20\/active-learning-in-mathematics-part-ii-levels-of-cognitive-demand\/><\/div>\n","protected":false},"author":76,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[197,27],"tags":[23,176,171,170],"class_list":["post-951","post","type-post","status-publish","format-standard","hentry","category-active-learning-in-mathematics-series-2015","category-classroom-practices","tag-active-learning","tag-active-learning-series-2015","tag-levels-of-cognitive-demand","tag-pnas"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p6C2AC-fl","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/posts\/951","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/users\/76"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/comments?post=951"}],"version-history":[{"count":27,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/posts\/951\/revisions"}],"predecessor-version":[{"id":997,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/posts\/951\/revisions\/997"}],"wp:attachment":[{"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/media?parent=951"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/categories?post=951"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.ams.org\/matheducation\/wp-json\/wp\/v2\/tags?post=951"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}