Session C16: Research on Student Reasoning and Problem Solving

Investigating the generalizability of a dual-process informed intervention strategy

An emerging body of research has shown that, even after research-based instruction, students who demonstrate correct conceptual understanding and reasoning on one task often fail to use the same knowledge and skills on related tasks. Observed inconsistencies can be accounted for by dual-process theories of reasoning (DPToR), which assert that human cognition relies on two thinking processes. The first, the heuristic process, is fast, intuitive, and automatic, while the second, the analytic process, is slow, effortful, and deliberate. We have recently found some success with a HW-based intervention that leverages DPToR to improve student reasoning about the terminal speed behavior of falling objects. In this talk, I will describe efforts to adapt the intervention strategy to closely-related content domains. I will describe the successes and failures of this approach, and draw conclusions about the generalizability of the DPToR-based intervention.   

Systematic spaced practices to promote error detection and override

Recent findings suggest that even those students who demonstrate relevant formal knowledge tend not to use it productively, especially on tasks that elicit intuitively appealing incorrect responses. Dual-process theories of reasoning suggest that to catch a mistake, reasoners must engage in error detection and override: recognize reasoning red flags, consider alternatives, and apply relevant knowledge to check their validity. It is, however, challenging for many novice physics learners to recognize what specific formal knowledge must be used as a criterion that needs to be satisfied for validating or rejecting a response. To help students develop skills necessary for error detection and override, we designed a sequence of systematic spaced practices in the context of Newton's 2nd law. We examined the effectiveness of this approach and identified specific factors that contribute to more productive engagement in error detection and override.   

Using dual-process theories as a lens to explore reasoning trajectories taken by students responding to physics questions

A growing body of research has found that students who have successfully applied relevant physics concepts and skills (mindware) to one physics question may perform inconsistently on an analogous question, even after research-based instruction. These inconsistencies can be explained through dual-process theories of reasoning (DPToR). According to DPToR, there are a variety of ways in which processes 1 and 2 may be engaged and interact while students are thinking about the physics question, and this leads to many possible reasoning trajectories. While increased attention to these trajectories may help improve the effectiveness of instructional interventions, existing (large N) methodologies do not allow for the systematic identification and characterization of such trajectories. In this study, students were first given a screening-target pair of questions, with the screening question providing an independent measure of mindware and the target question focusing on the same physics concept, but often eliciting intuitive, incorrect responses. The question pair was then followed by a set of metacognitive prompts in which students were asked to reflect on their reasoning on the target question. The focus of this ongoing work is to explore how student reasoning trajectories may be impacted by differences in the nature of the target question (and the associated intuitive models generated by process 1). In this talk, we present recent results and discuss implications for instructional materials.   

Using dual-process theories to investigate differences in student reasoning across questions that elicit different types of intuitive responses

Inconsistencies in student reasoning have been documented across a wide variety of physics contexts, and researchers have found that they frequently stem from the nature of human reasoning itself rather than from a lack of relevant knowledge and skills. A growing body of research has leveraged dual-process theories of reasoning (DPToR) as a framework to investigate these reasoning inconsistencies and as a guide for the development of interventions designed to better support reasoning. Much of this work focuses on questions that elicit intuitively appealing, incorrect answers. We have been designing and testing DPToR-aligned interventions that aim to help students productively engage in cognitive reflection and apply the knowledge and skills relevant to the correct solution, which they typically already possess. By administering analogous interventions to support student reasoning on questions in a variety of physics contexts, we seek to better understand how the specific nature of the intuitively appealing, incorrect response and the resources available to rationalize that response can impact how students engage with the intervention.  In this talk, we will present intervention results and discuss implications for research-based curriculum development.   

The Effects of a Problem-Solving Class on Student Persistence in STEM

Student persistence in STEM has been a focus of educational research, with both quantitative and qualitative methods being used to investigate patterns and causes of attrition. Some studies have used machine learning to predict a student's likelihood to persist given measurable classroom factors and incoming preparation, while others have framed persistence as a function of a student's social integration in the classroom. While these methods have provided insight into underlying causes of attrition in STEM, they have not investigated class structures or teaching methods that promote persistence. In this study we explore how an introductory active learning physics class using real world problem-solving (PS) had a positive impact on persistence for students at a large research-intensive university. Our findings showed that the one-year persistence rates for the PS course were 74% (fall) and 90% (spring), while the control class had a persistence rate of 64% and 78%, respectively. In spring, the PS persistence rate was significantly higher (p=0.037). The PS also had higher final grades and larger learning gains than the control despite lower incoming preparation. This study motivates future work to understand the mechanisms that promote student persistence in introductory physics.   

An Assessment of Expert-Like Problem-Solving Skills in Graduate Quantum Mechanics

Problem-solving is an important skill for physics graduate students; however, it is not clear to what extent this skill is explicitly taught in graduate coursework. To measure this skill, we developed an assessment of expert-like problem-solving skills (ELPSS)—which we define as decisions made with limited information—in graduate quantum mechanics. We first conducted semi-structured interviews with graduate physics instructors to identify ELPSS expected of first year graduate physics students. We then designed an assessment for graduate quantum mechanics which tests the expected ELPSS. The scoring rubric was developed using the consensus responses of experts (professional researchers in AMO and Condensed Matter) such that a higher score reflects a more expert-like response. We further tested the assessment with graduate and undergraduate students. We find evidence for the face validity of the assessment: experts receive the highest median scores of 78%, followed by graduate students with 33%, then undergraduates with 19%. This validation study indicates the potential for improved teaching of problem-solving skills in graduate physics coursework, which is the subject of an ongoing study.   

Requiring Proficiency While Allowing For Failure

Several factors influence student success and retention in STEM courses and/or degree programs including prior mathematics experience, stereotype threat, overall stress and mental load, etc.  When taken together, these factors can result in a distribution of pathways and associated rates required by students to achieve proficiency in the different content areas of their courses and degree programs, which, in turn, can affect both progression and retention in those programs. Our team has sought to better accommodate this diversity of student experience by developing a competency-based grading system for introductory physics.  In this system, students who do not demonstrate proficiency with an element of course content on an initial assessment have sufficient subsequent opportunities to do so.  This system also recognizes that coping with and learning from failure are essential to learning within STEM.  Finally, this assessment structure provides timely and individualized information about which content a student is struggling to master, enabling targeted interventions to help that student.  We show how the implementation of this grading system specifically improves student performance, particularly among groups that often pose a retention risk in science and engineering degree programs. We also discuss the benefits of this grading system for course and degree level assessment and propose future work to explore its use in a modified prerequisite system for other courses.   

Examining Student Reasoning at an HBCU: The Impact of Cultural Relevance

Physics education researchers agree that the field would benefit from investigations conducted with diverse student populations. This project examines physics teaching and student learning at a Historically Black University, thus contributing to a sparse body of research with this underrepresented population. We first presented questions designed to disentangle conceptual understanding, reasoning, and intuition reported in the literature to an HBCU classroom. The "screening" question was intended to probe whether students developed the necessary physics knowledge. The following "target" question required applying the same knowledge in a situation that elicits strongly appealing incorrect intuitive responses. To explore the importance of cultural relevance, the screening and target questions were re-worded to represent a more realistic situation. We compare student performance on the original and re-worded questions and discuss implications of making screening-target pairs more grounded.   

Progress towards a framework for Course-based undergraduate research experiences (CUREs) in Physics.

Undergraduate research experiences have been shown to offer significant benefits to students, particularly students from marginalized groups. However, the over-subscription of traditional research opportunities combined with systemic barriers prevent many from participating in these experiences. A promising alternative are course-based undergraduate research experiences, or CUREs. CUREs are shown to have similar outcomes to traditional undergraduate research experiences, while reducing the barriers to participation and providing an authentic research experience to an entire cohort of students . Within STEM disciplines, physics has been identified as underrepresented in CURE implementation and a field where CUREs could be added. The broad scope of our work is to identify the physics-specific challenges and opportunities for creating and sustaining CUREs in physics programs to be able to support more CUREs being offered. As a first step, we conducted a series of interviews with physics faculty members from multiple institutions to collect faculty opinions and experiences with learning outcomes, sustainability of curricular changes, and the hurdles and opportunities to sustainably engage them and their students in CUREs. We will present results and findings from these interviews and the next steps in developing the framework and a second CURE at our own institution.