-
Using Machine Learning to Identify the Most At-Risk Students in Physics Classes
Authors:
Jie Yang,
Seth DeVore,
Dona Hewagallage,
Paul Miller,
Qing X. Ryan,
John Stewart
Abstract:
Machine learning algorithms have recently been used to predict students' performance in an introductory physics class. The prediction model classified students as those likely to receive an A or B or students likely to receive a grade of C, D, F or withdraw from the class. Early prediction could better allow the direction of educational interventions and the allocation of educational resources. Ho…
▽ More
Machine learning algorithms have recently been used to predict students' performance in an introductory physics class. The prediction model classified students as those likely to receive an A or B or students likely to receive a grade of C, D, F or withdraw from the class. Early prediction could better allow the direction of educational interventions and the allocation of educational resources. However, the performance metrics used in that study become unreliable when used to classify whether a student would receive an A, B or C (the ABC outcome) or if they would receive a D, F or withdraw (W) from the class (the DFW outcome) because the outcome is substantially unbalanced with between 10\% to 20\% of the students receiving a D, F, or W. This work presents techniques to adjust the prediction models and alternate model performance metrics more appropriate for unbalanced outcome variables. These techniques were applied to three samples drawn from introductory mechanics classes at two institutions ($N=7184$, $1683$, and $926$). Applying the same methods as the earlier study produced a classifier that was very inaccurate, classifying only 16\% of the DFW cases correctly; tuning the model increased the DFW classification accuracy to 43\%. Using a combination of institutional and in-class data improved DFW accuracy to 53\% by the second week of class. As in the prior study, demographic variables such as gender, underrepresented minority status, first-generation college student status, and low socioeconomic status were not important variables in the final prediction models.
△ Less
Submitted 23 July, 2020;
originally announced July 2020.
-
Holistic framework to help students learn effectively from research-validated self-paced learning tools
Authors:
Emily Marshman,
Seth DeVore,
Chandralekha Singh
Abstract:
With limited time available in the classroom, e-learning tools can supplement in-class learning by providing opportunities for students to study and learn outside of class. Such tools can be especially helpful for students who lack adequate prior preparation. However, one critical issue is ensuring that students, especially those in need of additional help, engage with the tools as intended. Here…
▽ More
With limited time available in the classroom, e-learning tools can supplement in-class learning by providing opportunities for students to study and learn outside of class. Such tools can be especially helpful for students who lack adequate prior preparation. However, one critical issue is ensuring that students, especially those in need of additional help, engage with the tools as intended. Here we first discuss an empirical investigation in which students in a large algebra-based physics course were given opportunities to work through research-validated tutorials outside of class as self-study tools. Students were provided these optional tutorials after traditional instruction in relevant topics and were then given quizzes that included problems that were identical to the tutorial problems with regard to the physics principles involved but had different contexts. We find that students who worked through the tutorials as self-study tools struggled to transfer their learning to solve problems that used the same physics principles. On the other hand, students who worked on the tutorials in supervised, one-on-one situations performed significantly better than them. These empirical findings suggest that many introductory physics students may not engage effectively with self-paced learning tools unless they are provided additional incentives and support, e.g., to aid with self-regulation. Inspired by the empirical findings, we propose a holistic theoretical framework to help create learning environments in which students with diverse backgrounds are provided support to engage effectively with self-study tools.
△ Less
Submitted 6 July, 2020;
originally announced July 2020.
-
Extending Machine Learning to Predict Unbalanced Physics Course Outcomes
Authors:
Seth DeVore,
Jie Yang,
John Stewart
Abstract:
Machine learning algorithms have recently been used to classify students as those likely to receive an A or B or students likely to receive a C, D, or F in a physics class. The performance metrics used in that study become unreliable when the outcome variable is substantially unbalanced. This study seeks to further explored the classification of students who will receive a C, D, and F and extend t…
▽ More
Machine learning algorithms have recently been used to classify students as those likely to receive an A or B or students likely to receive a C, D, or F in a physics class. The performance metrics used in that study become unreliable when the outcome variable is substantially unbalanced. This study seeks to further explored the classification of students who will receive a C, D, and F and extend those methods to predicting whether a student will receive a D or F. The sample used for this work ($N=7184$) is substantially unbalanced with only 12\% of the students receiving a D or F. Applying the same methods as the previous study produced a classifier that was very inaccurate, classifying only 20\% of the D or F cases correctly. This study will focus on the random forest machine learning algorithm. By adjusting the random forest decision threshold, the correct classification rate of the D or F outcome rose to 46\%. This study also investigated the previous finding that demographic variables such as gender, underrepresented minority status, and first generation status had low variable importance for predicting class outcomes. Downsampling revealed that this was not the result of the underrepresentation of these students. An optimized classification model was constructed which predicted the D and F outcome with 46\% accuracy and C, D, and F outcome with 69\% accuracy; the accuracy of prediction of these outcomes is called "sensitivity" in the machine learning literature. Substantial variation was detected when this classification model was applied to predict the C, D, or F outcome for underrepresented demographic groups with 61\% sensitivity for women, 67\% for underrepresented minority students, and 78\% for first-generation students. Similar variation was observed for the D and F outcome.
△ Less
Submitted 5 February, 2020;
originally announced February 2020.
-
Multi-Dimensional Item Response Theory and the Force Concept Inventory
Authors:
John Stewart,
Cabot Zabriskie,
Seth DeVore,
Gay Stewart
Abstract:
Research on the test structure of the Force Concept Inventory (FCI) has largely been performed with exploratory methods such as factor analysis and cluster analysis. Multi-Dimensional Item Response Theory (MIRT) provides an alternative to traditional Exploratory Factor Analysis which allows statistical testing to identify the optimal number of factors. Application of MIRT to a sample of $N=4,716$…
▽ More
Research on the test structure of the Force Concept Inventory (FCI) has largely been performed with exploratory methods such as factor analysis and cluster analysis. Multi-Dimensional Item Response Theory (MIRT) provides an alternative to traditional Exploratory Factor Analysis which allows statistical testing to identify the optimal number of factors. Application of MIRT to a sample of $N=4,716$ FCI post-tests identified a 9-factor solution as optimal. Additional analysis showed that a substantial part of the identified factor structure resulted from the practice of using problem blocks and from pairs of similar questions. Applying MIRT to a reduced set of FCI items removing blocked items and repeated items produced a 6-factor solution; however, the factors had little relation the general structure of Newtonian mechanics. A theoretical model of the FCI was constructed from expert solutions and fit to the FCI by constraining the MIRT parameter matrix to the theoretical model. Variations on the theoretical model were then explored to identify an optimal model. The optimal model supported the differentiation of Newton's 1st and 2nd law; of one-dimensional and three-dimensional kinematics; and of the principle of the addition of forces from Newton's 2nd law. The model suggested by the authors of the FCI was also fit; the optimal MIRT model was statistically superior.
△ Less
Submitted 6 March, 2018;
originally announced March 2018.
-
The challenge of engaging all students via self-paced interactive e-learning tutorials for introductory physics
Authors:
Seth DeVore,
Emily Marshman,
Chandralekha Singh
Abstract:
As research-based self-paced e-learning tools become increasingly available, a critical issue educators encounter is implementing strategies to ensure that all students engage with them as intended. Here, we discuss the effectiveness of research-based e-learning tutorials as self-paced learning tools in large enrollment brick and mortar introductory physics courses. These interactive tutorials wer…
▽ More
As research-based self-paced e-learning tools become increasingly available, a critical issue educators encounter is implementing strategies to ensure that all students engage with them as intended. Here, we discuss the effectiveness of research-based e-learning tutorials as self-paced learning tools in large enrollment brick and mortar introductory physics courses. These interactive tutorials were developed via research in physics education and were found to be effective for a diverse group of introductory physics students in one-on-one implementation. Instructors encouraged the use of these self-paced tools in a self-paced learning environment by telling students that they would be helpful for solving the assigned homework problems and that the underlying physics principles in the tutorial problems would be similar to those in the in-class quizzes (which we call paired problems). We find that many students, who struggled in the courses in which these adaptive e-learning tutorials were assigned as a self-study tool, performed poorly on the paired problems. In contrast, a majority of student volunteers in one-on-one implementation greatly benefited from the tutorials and performed well on the paired problems. This suggests that many students enrolled in introductory physics courses did not effectively engage with the self-paced tutorials outside of class and may have only used them superficially. The findings suggest that many students in need of out-of-class remediation via self-paced learning tools may have difficulty motivating themselves and may lack the self-regulation and time-management skills to engage effectively with tools specially designed to help them learn at their own pace. We conclude by proposing a theoretical framework to help students with diverse prior preparations engage effectively with self-study tools.
△ Less
Submitted 1 June, 2016;
originally announced June 2016.
-
Development and Evaluation of a Tutorial to Improve Students' Understanding of a Lock-in amplifier
Authors:
Seth DeVore,
Alexandre Gauthier,
Jeremy Levy,
Chandralekha Singh
Abstract:
A lock-in amplifier is a versatile instrument frequently used in physics research. However, many students struggle with the basic operating principles of a lock-in amplifier which can lead to a variety of difficulties. To improve students' understanding, we have been developing and evaluating a research-based tutorial which makes use of a computer simulation of a lock-in amplifier. The tutorial is…
▽ More
A lock-in amplifier is a versatile instrument frequently used in physics research. However, many students struggle with the basic operating principles of a lock-in amplifier which can lead to a variety of difficulties. To improve students' understanding, we have been developing and evaluating a research-based tutorial which makes use of a computer simulation of a lock-in amplifier. The tutorial is based on a field-tested approach in which students realize their difficulties after predicting the outcome of simulated experiments involving a lock-in amplifier and check their predictions using the simulated lock-in amplifier. Then, the tutorial provides guidance and strives to help students develop a coherent understanding of the basics of a lock-in amplifier. The tutorial development involved interviews with physics faculty members and graduate students and iteration of many versions of the tutorial with professors and graduate students. The student difficulties with lock-in amplifiers and the development and assessment of the research-based tutorial to help students develop a functional understanding of this device are discussed.
△ Less
Submitted 6 January, 2016;
originally announced January 2016.
-
Interactive Learning Tutorial on Quantum Key Distribution
Authors:
Seth DeVore,
Chandralekha Singh
Abstract:
We describe the development and in-class evaluation of a Quantum Interactive Learning Tutorial (QuILT) on quantum key distribution, a context which involves an exciting application of quantum mechanics. The protocol used in the QuILT described here uses single photons with non-orthogonal polarization states to generate a random shared key over a public channel for encrypting and decrypting informa…
▽ More
We describe the development and in-class evaluation of a Quantum Interactive Learning Tutorial (QuILT) on quantum key distribution, a context which involves an exciting application of quantum mechanics. The protocol used in the QuILT described here uses single photons with non-orthogonal polarization states to generate a random shared key over a public channel for encrypting and decrypting information. The QuILT strives to help upper-level undergraduate students learn quantum mechanics using a simple two state system. It actively engages students in the learning process and helps them build links between the formalism and the conceptual aspects of quantum physics without compromising the technical content. The in-class evaluation suggests that the validated QuILT is helpful in improving students' understanding of relevant concepts.
△ Less
Submitted 17 June, 2020; v1 submitted 6 January, 2016;
originally announced January 2016.
-
Improving Students' Understanding of Lock-In Amplifiers
Authors:
Seth DeVore,
Alexandre Gauthier,
Jeremy Levy,
Chandralekha Singh
Abstract:
The lock-in amplifier is a versatile instrument frequently used in physics research. However, many students struggle with the basic operating principles of a lock-in amplifier which can lead to a variety of difficulties. To improve students' understanding, we have been developing and evaluating a research-based tutorial which makes use of a computer simulation of a lock-in amplifier. The tutorial…
▽ More
The lock-in amplifier is a versatile instrument frequently used in physics research. However, many students struggle with the basic operating principles of a lock-in amplifier which can lead to a variety of difficulties. To improve students' understanding, we have been developing and evaluating a research-based tutorial which makes use of a computer simulation of a lock-in amplifier. The tutorial allows students to realize their conceptual difficulties by comparing their predictions about how a lock-in amplifier will behave with the outcome of simulated experiments. After being confronted with discrepancies between their predictions and the lock-in amplifier results, the tutorial targets the common difficulties that students have and leads them to develop a deeper understanding of the principles of lock-in amplification.
△ Less
Submitted 6 January, 2016;
originally announced January 2016.
-
Improving Students' Understanding of Lock-in Amplifiers
Authors:
Seth DeVore,
Alexandre Gauthier,
Jeremy Levy,
Chandralekha Singh
Abstract:
A lock-in amplifier is a versatile instrument frequently used in physics research. However, many students struggle with the basic operating principles of a lock-in amplifier which can lead to a variety of difficulties. To improve students' understanding, we have been developing and evaluating a research-based tutorial which makes use of a computer simulation of a lock-in amplifier. The tutorial is…
▽ More
A lock-in amplifier is a versatile instrument frequently used in physics research. However, many students struggle with the basic operating principles of a lock-in amplifier which can lead to a variety of difficulties. To improve students' understanding, we have been developing and evaluating a research-based tutorial which makes use of a computer simulation of a lock-in amplifier. The tutorial is based on a field-tested approach in which students realize their difficulties after predicting the outcome of simulated experiments involving a lock-in amplifier and check their predictions using the simulated lock-in amplifier. Then, the tutorial guides and helps students develop a coherent understanding of the basics of a lock-in amplifier. The tutorial development involved interviews with physics faculty members and graduate students and iteration of many versions of the tutorial with professors and graduate students. The student difficulties and the development and assessment of the research-based tutorial are discussed.
△ Less
Submitted 4 January, 2016;
originally announced January 2016.
-
Development of an interactive tutorial on quantum key distribution
Authors:
Seth DeVore,
Chandralekha Singh
Abstract:
We describe the development of a Quantum Interactive Learning Tutorial (QuILT) on quantum key distribution, a context which involves a practical application of quantum mechanics. The QuILT helps upper-level undergraduate students learn quantum mechanics using a simple two state system and was developed based upon the findings of cognitive research and physics education research. One protocol used…
▽ More
We describe the development of a Quantum Interactive Learning Tutorial (QuILT) on quantum key distribution, a context which involves a practical application of quantum mechanics. The QuILT helps upper-level undergraduate students learn quantum mechanics using a simple two state system and was developed based upon the findings of cognitive research and physics education research. One protocol used in the QuILT involves generating a random shared key over a public channel for encrypting and decrypting information using single photons with non-orthogonal polarization states, and another protocol makes use of two entangled spin-1/2 particles. The QuILT uses a guided approach and focuses on helping students build links between the formalism and conceptual aspects of quantum physics without compromising the technical content. We also discuss findings from a preliminary in-class evaluation.
△ Less
Submitted 4 January, 2016;
originally announced January 2016.
-
Analog Optical Computing Primitives in Silicon Photonics
Authors:
Yunshan Jiang,
Peter T. S. DeVore,
Bahram Jalali
Abstract:
Optical computing accelerators may help alleviate bandwidth and power consumption bottlenecks in electronics. We show an approach to implementing logarithmic-type analog co-processors in silicon photonics and use it to perform the exponentiation operation. The function is realized by exploiting nonlinear-absorption-enhanced Raman amplification saturation in a silicon waveguide.
Optical computing accelerators may help alleviate bandwidth and power consumption bottlenecks in electronics. We show an approach to implementing logarithmic-type analog co-processors in silicon photonics and use it to perform the exponentiation operation. The function is realized by exploiting nonlinear-absorption-enhanced Raman amplification saturation in a silicon waveguide.
△ Less
Submitted 2 October, 2015;
originally announced October 2015.
-
Demonstration of Vpi Reduction in Electrooptic Modulators using Modulation Instability
Authors:
David Borlaug,
Peter T. S. DeVore,
Ali Rostami,
Ozdal Boyraz,
Bahram Jalali
Abstract:
Reduction in the operating voltage of electrooptic modulators is needed in order to remove the bottleneck in the flow of data between electronics and optical interconnects. We report experimental demonstration of a 10 fold reduction in an electrooptic modulator's half-wave voltage up to 50 GHz. This was achieved by employing optical sideband-only amplification.
Reduction in the operating voltage of electrooptic modulators is needed in order to remove the bottleneck in the flow of data between electronics and optical interconnects. We report experimental demonstration of a 10 fold reduction in an electrooptic modulator's half-wave voltage up to 50 GHz. This was achieved by employing optical sideband-only amplification.
△ Less
Submitted 28 July, 2014;
originally announced July 2014.
-
Enhancing the Electrooptic Effect Using Modulation Instability
Authors:
Peter T. S. DeVore,
David Borlaug,
Bahram Jalali
Abstract:
As electronic operating frequencies increase toward the terahertz regime, new electrooptic modulators capable of low-voltage high-frequency operation must be developed to provide the necessary optical interconnects. This letter presents a new concept that exploits modulation instability to enhance the intrinsically weak electrooptic effect, χ^(2). Simulations demonstrate more than 50 times enhance…
▽ More
As electronic operating frequencies increase toward the terahertz regime, new electrooptic modulators capable of low-voltage high-frequency operation must be developed to provide the necessary optical interconnects. This letter presents a new concept that exploits modulation instability to enhance the intrinsically weak electrooptic effect, χ^(2). Simulations demonstrate more than 50 times enhancement of electrooptic effect at millimeter wave frequencies leading to a substantial reduction in the required modulation voltage.
△ Less
Submitted 13 February, 2013;
originally announced February 2013.
