-
Improvements & Evaluations on the MLCommons CloudMask Benchmark
Authors:
Varshitha Chennamsetti,
Laiba Mehnaz,
Dan Zhao,
Banani Ghosh,
Sergey V. Samsonau
Abstract:
In this paper, we report the performance benchmarking results of deep learning models on MLCommons' Science cloud-masking benchmark using a high-performance computing cluster at New York University (NYU): NYU Greene. MLCommons is a consortium that develops and maintains several scientific benchmarks that can benefit from developments in AI. We provide a description of the cloud-masking benchmark t…
▽ More
In this paper, we report the performance benchmarking results of deep learning models on MLCommons' Science cloud-masking benchmark using a high-performance computing cluster at New York University (NYU): NYU Greene. MLCommons is a consortium that develops and maintains several scientific benchmarks that can benefit from developments in AI. We provide a description of the cloud-masking benchmark task, updated code, and the best model for this benchmark when using our selected hyperparameter settings. Our benchmarking results include the highest accuracy achieved on the NYU system as well as the average time taken for both training and inference on the benchmark across several runs/seeds. Our code can be found on GitHub. MLCommons team has been kept informed about our progress and may use the developed code for their future work.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
MLCommons Cloud Masking Benchmark with Early Stopping
Authors:
Varshitha Chennamsetti,
Gregor von Laszewski,
Ruochen Gu,
Laiba Mehnaz,
Juri Papay,
Samuel Jackson,
Jeyan Thiyagalingam,
Sergey V. Samsonau,
Geoffrey C. Fox
Abstract:
In this paper, we report on work performed for the MLCommons Science Working Group on the cloud masking benchmark. MLCommons is a consortium that develops and maintains several scientific benchmarks that aim to benefit developments in AI. The benchmarks are conducted on the High Performance Computing (HPC) Clusters of New York University and University of Virginia, as well as a commodity desktop.…
▽ More
In this paper, we report on work performed for the MLCommons Science Working Group on the cloud masking benchmark. MLCommons is a consortium that develops and maintains several scientific benchmarks that aim to benefit developments in AI. The benchmarks are conducted on the High Performance Computing (HPC) Clusters of New York University and University of Virginia, as well as a commodity desktop. We provide a description of the cloud masking benchmark, as well as a summary of our submission to MLCommons on the benchmark experiment we conducted. It includes a modification to the reference implementation of the cloud masking benchmark enabling early stopping. This benchmark is executed on the NYU HPC through a custom batch script that runs the various experiments through the batch queuing system while allowing for variation on the number of epochs trained. Our submission includes the modified code, a custom batch script to modify epochs, documentation, and the benchmark results. We report the highest accuracy (scientific metric) and the average time taken (performance metric) for training and inference that was achieved on NYU HPC Greene. We also provide a comparison of the compute capabilities between different systems by running the benchmark for one epoch. Our submission can be found in a Globus repository that is accessible to MLCommons Science Working Group.
△ Less
Submitted 30 May, 2024; v1 submitted 11 December, 2023;
originally announced January 2024.
-
Artificial Intelligence for Scientific Research: Authentic Research Education Framework
Authors:
Sergey V Samsonau,
Aziza Kurbonova,
Lu Jiang,
Hazem Lashen,
Jiamu Bai,
Theresa Merchant,
Ruoxi Wang,
Laiba Mehnaz,
Zecheng Wang,
Ishita Patil
Abstract:
We report a framework that enables the wide adoption of authentic research educational methodology at various schools by addressing common barriers. The guiding principles we present were applied to implement a program in which teams of students with complementary skills develop useful artificial intelligence (AI) solutions for researchers in natural sciences. To accomplish this, we work with rese…
▽ More
We report a framework that enables the wide adoption of authentic research educational methodology at various schools by addressing common barriers. The guiding principles we present were applied to implement a program in which teams of students with complementary skills develop useful artificial intelligence (AI) solutions for researchers in natural sciences. To accomplish this, we work with research laboratories that reveal/specify their needs, and then our student teams work on the discovery, design, and development of an AI solution for unique problems using a consulting-like arrangement. To date, our group has been operating at New York University (NYU) for seven consecutive semesters, has engaged more than a hundred students, ranging from first-year college students to master's candidates, and has worked with more than twenty projects and collaborators. While creating education benefits for students, our approach also directly benefits scientists, who get an opportunity to evaluate the usefulness of machine learning for their specific needs.
△ Less
Submitted 7 July, 2024; v1 submitted 19 September, 2022;
originally announced October 2022.
-
Preparing students for authentic research through open-inquiry laboratory courses
Authors:
Roxanne P. Spencer,
Sergey V. Samsonau
Abstract:
Authentic research as a method of teaching science is gaining popularity in high schools and colleges. To make this research experience most efficient, students need adequate preparation in traditional science courses. Existing materials available for inquiry-based teaching methods are not intended to prepare a student for independent research. We discuss principles for organization of laboratory…
▽ More
Authentic research as a method of teaching science is gaining popularity in high schools and colleges. To make this research experience most efficient, students need adequate preparation in traditional science courses. Existing materials available for inquiry-based teaching methods are not intended to prepare a student for independent research. We discuss principles for organization of laboratory classes with this goal in mind and provide examples that we have implemented for algebra-based physics, calculus-based physics, and honors chemistry for high school students.
△ Less
Submitted 9 August, 2018; v1 submitted 8 August, 2018;
originally announced August 2018.
-
Computer simulations combined with experiments for calculus-based physics laboratory course
Authors:
Sergey V. Samsonau
Abstract:
This paper is presenting a set of laboratory classes to be taught as a part of a 1-year calculus-based physics class. It is composed out of 7 modules designed to bring together experiments and computer simulations (numerical simulations). Each module uses both simulations and experiments to address a phenomenon under study, and lasts for 3 weeks (21 weeks total for the whole set). Wolfram Mathemat…
▽ More
This paper is presenting a set of laboratory classes to be taught as a part of a 1-year calculus-based physics class. It is composed out of 7 modules designed to bring together experiments and computer simulations (numerical simulations). Each module uses both simulations and experiments to address a phenomenon under study, and lasts for 3 weeks (21 weeks total for the whole set). Wolfram Mathematica is used for computer simulations. Topics are: Motion with a drag, Pendulum with an arbitrary amplitude, Magnus effect, Centripetal force acting on a pendulum, A ball rolling from paths of various shapes, Doppler shift and Fourier transformation, Equipotential lines. These laboratory classes were taught for 2 years at Princeton International School of Mathematics and Science.
△ Less
Submitted 22 May, 2018; v1 submitted 24 April, 2018;
originally announced April 2018.
