-
Data fusion of complementary data sources using Machine Learning enables higher accuracy Solar Resource Maps
Authors:
J Rabault,
ML Sætra,
A Dobler,
S Eastwood,
E Berge
Abstract:
In the present work, we collect solar irradiance and atmospheric condition data from several products, obtained from both numerical models (ERA5 and NORA3) and satellite observations (CMSAF-SARAH3). We then train simple supervised Machine Learning (ML) data fusion models, using these products as predictors and direct in-situ Global Horizontal Irradiance (GHI) measurements over Norway as ground-tru…
▽ More
In the present work, we collect solar irradiance and atmospheric condition data from several products, obtained from both numerical models (ERA5 and NORA3) and satellite observations (CMSAF-SARAH3). We then train simple supervised Machine Learning (ML) data fusion models, using these products as predictors and direct in-situ Global Horizontal Irradiance (GHI) measurements over Norway as ground-truth. We show that combining these products by applying our trained ML models provides a GHI estimate that is significantly more accurate than that obtained from any product taken individually. Using the trained models, we generate a 30-year ML-corrected map of GHI over Norway, which we release as a new open data product. Our ML-based data fusion methodology could be applied, after suitable training and input data selection, to any geographic area on Earth.
△ Less
Submitted 22 January, 2025; v1 submitted 8 January, 2025;
originally announced January 2025.
-
Bias Correction of Operational Storm Surge Forecasts Using Neural Networks
Authors:
Paulina Tedesco,
Jean Rabault,
Martin Lilleeng Sætra,
Nils Melsom Kristensen,
Ole Johan Aarnes,
Øyvind Breivik,
Cecilie Mauritzen,
Øyvind Sætra
Abstract:
Storm surges can give rise to extreme floods in coastal areas. The Norwegian Meteorological Institute produces 120-hour regional operational storm surge forecasts along the coast of Norway based on the Regional Ocean Modeling System (ROMS), using a model setup called Nordic4-SS. Despite advances in the development of models and computational capabilities, forecast errors remain large enough to imp…
▽ More
Storm surges can give rise to extreme floods in coastal areas. The Norwegian Meteorological Institute produces 120-hour regional operational storm surge forecasts along the coast of Norway based on the Regional Ocean Modeling System (ROMS), using a model setup called Nordic4-SS. Despite advances in the development of models and computational capabilities, forecast errors remain large enough to impact response measures and issued alerts, in particular, during the strongest events. Reducing these errors will positively impact the efficiency of the warning systems while minimizing efforts and resources spent on mitigation. Here, we investigate how forecasts can be improved with residual learning, i.e., training data-driven models to predict the residuals in forecasts from Nordic4-SS. A simple error mapping technique and a more sophisticated Neural Network (NN) method are tested. Using the NN residual correction method, the Root Mean Square Error in the Oslo Fjord is reduced by 36% for lead times of one hour and 9% for 24 hours. Therefore, the residual NN method is a promising direction for correcting storm surge forecasts, especially on short timescales. Moreover, it is well adapted to being deployed operationally, as i) the correction is applied on top of the existing model and requires no changes to it, ii) all predictors used for NN inference are already available operationally, iii) prediction by the NNs is very fast, typically a few seconds per station, and iv) the NN correction can be provided to a human expert who may inspect it, compare it with the model output, and see how much correction is brought by the NN, allowing to capitalize on human expertise as a quality validation of the NN output. While no changes to the hydrodynamic model are necessary to calibrate the neural networks, they are specific to a given model and must be recalibrated when the numerical models are updated.
△ Less
Submitted 18 December, 2023; v1 submitted 2 January, 2023;
originally announced January 2023.
-
GPU Computing with Python: Performance, Energy Efficiency and Usability
Authors:
Håvard H. Holm,
André R. Brodtkorb,
Martin L. Sætra
Abstract:
In this work, we examine the performance, energy efficiency and usability when using Python for developing HPC codes running on the GPU. We investigate the portability of performance and energy efficiency between CUDA and OpenCL; between GPU generations; and between low-end, mid-range and high-end GPUs. Our findings show that the impact of using Python is negligible for our applications, and furth…
▽ More
In this work, we examine the performance, energy efficiency and usability when using Python for developing HPC codes running on the GPU. We investigate the portability of performance and energy efficiency between CUDA and OpenCL; between GPU generations; and between low-end, mid-range and high-end GPUs. Our findings show that the impact of using Python is negligible for our applications, and furthermore, CUDA and OpenCL applications tuned to an equivalent level can in many cases obtain the same computational performance. Our experiments show that performance in general varies more between different GPUs than between using CUDA and OpenCL. We also show that tuning for performance is a good way of tuning for energy efficiency, but that specific tuning is needed to obtain optimal energy efficiency.
△ Less
Submitted 5 December, 2019;
originally announced December 2019.
-
Massively Parallel Implicit Equal-Weights Particle Filter for Ocean Drift Trajectory Forecasting
Authors:
Håvard Heitlo Holm,
Martin Lilleeng Sætra,
Peter Jan van Leeuwen
Abstract:
Forecasting ocean drift trajectories are important for many applications, including search and rescue operations, oil spill cleanup and iceberg risk mitigation. In an operational setting, forecasts of drift trajectories are produced based on computationally demanding forecasts of three-dimensional ocean currents. Herein, we investigate a complementary approach for shorter time scales by using a re…
▽ More
Forecasting ocean drift trajectories are important for many applications, including search and rescue operations, oil spill cleanup and iceberg risk mitigation. In an operational setting, forecasts of drift trajectories are produced based on computationally demanding forecasts of three-dimensional ocean currents. Herein, we investigate a complementary approach for shorter time scales by using a recent state-of-the-art implicit equal-weights particle filter applied to a simplified ocean model. To achieve this, we present a new algorithmic design for a data-assimilation system in which all components - including the model, model errors, and particle filter - take advantage of massively parallel compute architectures, such as graphical processing units. Faster computations can enable in-situ and ad-hoc model runs for emergency management, and larger ensembles for better uncertainty quantification. Using a challenging test case with near-realistic chaotic instabilities, we run data-assimilation experiments based on synthetic observations from drifting and moored buoys, and analyse the trajectory forecasts for the drifters. Our results show that even sparse drifter observations are sufficient to significantly improve short-term drift forecasts up to twelve hours. With equidistant moored buoys observing only 0.1% of the state space, the ensemble gives an accurate description of the true state after data assimilation followed by a high-quality probabilistic forecast.
△ Less
Submitted 2 October, 2019;
originally announced October 2019.
