-
Improvements & Evaluations on the MLCommons CloudMask Benchmark
Authors:
Varshitha Chennamsetti,
Laiba Mehnaz,
Dan Zhao,
Banani Ghosh,
Sergey V. Samsonau
Abstract:
In this paper, we report the performance benchmarking results of deep learning models on MLCommons' Science cloud-masking benchmark using a high-performance computing cluster at New York University (NYU): NYU Greene. MLCommons is a consortium that develops and maintains several scientific benchmarks that can benefit from developments in AI. We provide a description of the cloud-masking benchmark t…
▽ More
In this paper, we report the performance benchmarking results of deep learning models on MLCommons' Science cloud-masking benchmark using a high-performance computing cluster at New York University (NYU): NYU Greene. MLCommons is a consortium that develops and maintains several scientific benchmarks that can benefit from developments in AI. We provide a description of the cloud-masking benchmark task, updated code, and the best model for this benchmark when using our selected hyperparameter settings. Our benchmarking results include the highest accuracy achieved on the NYU system as well as the average time taken for both training and inference on the benchmark across several runs/seeds. Our code can be found on GitHub. MLCommons team has been kept informed about our progress and may use the developed code for their future work.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
MLCommons Cloud Masking Benchmark with Early Stopping
Authors:
Varshitha Chennamsetti,
Gregor von Laszewski,
Ruochen Gu,
Laiba Mehnaz,
Juri Papay,
Samuel Jackson,
Jeyan Thiyagalingam,
Sergey V. Samsonau,
Geoffrey C. Fox
Abstract:
In this paper, we report on work performed for the MLCommons Science Working Group on the cloud masking benchmark. MLCommons is a consortium that develops and maintains several scientific benchmarks that aim to benefit developments in AI. The benchmarks are conducted on the High Performance Computing (HPC) Clusters of New York University and University of Virginia, as well as a commodity desktop.…
▽ More
In this paper, we report on work performed for the MLCommons Science Working Group on the cloud masking benchmark. MLCommons is a consortium that develops and maintains several scientific benchmarks that aim to benefit developments in AI. The benchmarks are conducted on the High Performance Computing (HPC) Clusters of New York University and University of Virginia, as well as a commodity desktop. We provide a description of the cloud masking benchmark, as well as a summary of our submission to MLCommons on the benchmark experiment we conducted. It includes a modification to the reference implementation of the cloud masking benchmark enabling early stopping. This benchmark is executed on the NYU HPC through a custom batch script that runs the various experiments through the batch queuing system while allowing for variation on the number of epochs trained. Our submission includes the modified code, a custom batch script to modify epochs, documentation, and the benchmark results. We report the highest accuracy (scientific metric) and the average time taken (performance metric) for training and inference that was achieved on NYU HPC Greene. We also provide a comparison of the compute capabilities between different systems by running the benchmark for one epoch. Our submission can be found in a Globus repository that is accessible to MLCommons Science Working Group.
△ Less
Submitted 30 May, 2024; v1 submitted 11 December, 2023;
originally announced January 2024.
-
Artificial Intelligence for Scientific Research: Authentic Research Education Framework
Authors:
Sergey V Samsonau,
Aziza Kurbonova,
Lu Jiang,
Hazem Lashen,
Jiamu Bai,
Theresa Merchant,
Ruoxi Wang,
Laiba Mehnaz,
Zecheng Wang,
Ishita Patil
Abstract:
We report a framework that enables the wide adoption of authentic research educational methodology at various schools by addressing common barriers. The guiding principles we present were applied to implement a program in which teams of students with complementary skills develop useful artificial intelligence (AI) solutions for researchers in natural sciences. To accomplish this, we work with rese…
▽ More
We report a framework that enables the wide adoption of authentic research educational methodology at various schools by addressing common barriers. The guiding principles we present were applied to implement a program in which teams of students with complementary skills develop useful artificial intelligence (AI) solutions for researchers in natural sciences. To accomplish this, we work with research laboratories that reveal/specify their needs, and then our student teams work on the discovery, design, and development of an AI solution for unique problems using a consulting-like arrangement. To date, our group has been operating at New York University (NYU) for seven consecutive semesters, has engaged more than a hundred students, ranging from first-year college students to master's candidates, and has worked with more than twenty projects and collaborators. While creating education benefits for students, our approach also directly benefits scientists, who get an opportunity to evaluate the usefulness of machine learning for their specific needs.
△ Less
Submitted 7 July, 2024; v1 submitted 19 September, 2022;
originally announced October 2022.
