-
Dictionary learning -- from local towards global and adaptive
Authors:
Marie Christine Pali,
Karin Schnass
Abstract:
This paper studies the convergence behaviour of dictionary learning via the Iterative Thresholding and K-residual Means (ITKrM) algorithm. On one hand it is proved that ITKrM is a contraction under much more relaxed conditions than previously necessary. On the other hand it is shown that there seem to exist stable fixed points that do not correspond to the generating dictionary, which can be chara…
▽ More
This paper studies the convergence behaviour of dictionary learning via the Iterative Thresholding and K-residual Means (ITKrM) algorithm. On one hand it is proved that ITKrM is a contraction under much more relaxed conditions than previously necessary. On the other hand it is shown that there seem to exist stable fixed points that do not correspond to the generating dictionary, which can be characterised as very coherent. Based on an analysis of the residuals using these bad dictionaries, replacing coherent atoms with carefully designed replacement candidates is proposed. In experiments on synthetic data this outperforms random or no replacement and always leads to full dictionary recovery. Finally the question how to learn dictionaries without knowledge of the correct dictionary size and sparsity level is addressed. Decoupling the replacement strategy of coherent or unused atoms into pruning and adding, and slowly carefully increasing the sparsity level, leads to an adaptive version of ITKrM. In several experiments this adaptive dictionary learning algorithm is shown to recover a generating dictionary from randomly initialised dictionaries of various sizes on synthetic data and to learn meaningful dictionaries on image data.
△ Less
Submitted 21 April, 2021; v1 submitted 19 April, 2018;
originally announced April 2018.
-
Short-term solar irradiance and irradiation forecasts via different time series techniques: A preliminary study
Authors:
Cédric Join,
Cyril Voyant,
Michel Fliess,
Marc Muselli,
Marie Laure Nivet,
Christophe Paoli,
Frédéric Chaxel
Abstract:
This communication is devoted to solar irradiance and irradiation short-term forecasts, which are useful for electricity production. Several different time series approaches are employed. Our results and the corresponding numerical simulations show that techniques which do not need a large amount of historical data behave better than those which need them, especially when those data are quite nois…
▽ More
This communication is devoted to solar irradiance and irradiation short-term forecasts, which are useful for electricity production. Several different time series approaches are employed. Our results and the corresponding numerical simulations show that techniques which do not need a large amount of historical data behave better than those which need them, especially when those data are quite noisy.
△ Less
Submitted 26 September, 2014;
originally announced September 2014.
-
Meteorological time series forecasting with pruned multi-layer perceptron and 2-stage Levenberg-Marquardt method
Authors:
Cyril Voyant,
Wani W. Tamas,
Marie Laure Nivet,
Gilles Notton,
Christophe Paoli,
Aurélia Balu,
Marc Muselli
Abstract:
A Multi-Layer Perceptron (MLP) defines a family of artificial neural networks often used in TS modeling and forecasting. Because of its "black box" aspect, many researchers refuse to use it. Moreover, the optimization (often based on the exhaustive approach where "all" configurations are tested) and learning phases of this artificial intelligence tool (often based on the Levenberg-Marquardt algori…
▽ More
A Multi-Layer Perceptron (MLP) defines a family of artificial neural networks often used in TS modeling and forecasting. Because of its "black box" aspect, many researchers refuse to use it. Moreover, the optimization (often based on the exhaustive approach where "all" configurations are tested) and learning phases of this artificial intelligence tool (often based on the Levenberg-Marquardt algorithm; LMA) are weaknesses of this approach (exhaustively and local minima). These two tasks must be repeated depending on the knowledge of each new problem studied, making the process, long, laborious and not systematically robust. In this paper a pruning process is proposed. This method allows, during the training phase, to carry out an inputs selecting method activating (or not) inter-nodes connections in order to verify if forecasting is improved. We propose to use iteratively the popular damped least-squares method to activate inputs and neurons. A first pass is applied to 10% of the learning sample to determine weights significantly different from 0 and delete other. Then a classical batch process based on LMA is used with the new MLP. The validation is done using 25 measured meteorological TS and cross-comparing the prediction results of the classical LMA and the 2-stage LMA.
△ Less
Submitted 8 July, 2014;
originally announced July 2014.
-
Meteorological time series forecasting based on MLP modelling using heterogeneous transfer functions
Authors:
Cyril Voyant,
Marie Laure Nivet,
Christophe Paoli,
Marc Muselli,
Gilles Notton
Abstract:
In this paper, we propose to study four meteorological and seasonal time series coupled with a multi-layer perceptron (MLP) modeling. We chose to combine two transfer functions for the nodes of the hidden layer, and to use a temporal indicator (time index as input) in order to take into account the seasonal aspect of the studied time series. The results of the prediction concern two years of measu…
▽ More
In this paper, we propose to study four meteorological and seasonal time series coupled with a multi-layer perceptron (MLP) modeling. We chose to combine two transfer functions for the nodes of the hidden layer, and to use a temporal indicator (time index as input) in order to take into account the seasonal aspect of the studied time series. The results of the prediction concern two years of measurements and the learning step, eight independent years. We show that this methodology can improve the accuracy of meteorological data estimation compared to a classical MLP modelling with a homogenous transfer function.
△ Less
Submitted 29 April, 2014;
originally announced April 2014.
-
Numerical weather prediction or stochastic modeling: an objective criterion of choice for the global radiation forecasting
Authors:
Cyril Voyant,
Gilles Notton,
Christophe Paoli,
Marie Laure Nivet,
Marc Muselli,
Kahina Dahmani
Abstract:
Numerous methods exist and were developed for global radiation forecasting. The two most popular types are the numerical weather predictions (NWP) and the predictions using stochastic approaches. We propose to compute a parameter noted constructed in part from the mutual information which is a quantity that measures the mutual dependence of two variables. Both of these are calculated with the obje…
▽ More
Numerous methods exist and were developed for global radiation forecasting. The two most popular types are the numerical weather predictions (NWP) and the predictions using stochastic approaches. We propose to compute a parameter noted constructed in part from the mutual information which is a quantity that measures the mutual dependence of two variables. Both of these are calculated with the objective to establish the more relevant method between NWP and stochastic models concerning the current problem.
△ Less
Submitted 22 January, 2014;
originally announced January 2014.
-
Time series modeling and large scale global solar radiation forecasting from geostationary satellites data
Authors:
Cyril Voyant,
Pierrick Haurant,
Marc Muselli,
Christophe Paoli,
Marie Laure Nivet
Abstract:
When a territory is poorly instrumented, geostationary satellites data can be useful to predict global solar radiation. In this paper, we use geostationary satellites data to generate 2-D time series of solar radiation for the next hour. The results presented in this paper relate to a particular territory, the Corsica Island, but as data used are available for the entire surface of the globe, our…
▽ More
When a territory is poorly instrumented, geostationary satellites data can be useful to predict global solar radiation. In this paper, we use geostationary satellites data to generate 2-D time series of solar radiation for the next hour. The results presented in this paper relate to a particular territory, the Corsica Island, but as data used are available for the entire surface of the globe, our method can be easily exploited to another place. Indeed 2-D hourly time series are extracted from the HelioClim-3 surface solar irradiation database treated by the Heliosat-2 model. Each point of the map have been used as training data and inputs of artificial neural networks (ANN) and as inputs for two persistence models (scaled or not). Comparisons between these models and clear sky estimations were proceeded to evaluate the performances. We found a normalized root mean square error (nRMSE) close to 16.5% for the two best predictors (scaled persistence and ANN) equivalent to 35-45% related to ground measurements. Finally in order to validate our 2-D predictions maps, we introduce a new error metric called the gamma index which is a criterion for comparing data from two matrixes in medical physics. As first results, we found that in winter and spring, scaled persistence gives the best results (gamma index test passing rate is respectively 67.7% and 86%), in autumn simple persistence is the best predictor (95.3%) and ANN is the best in summer (99.8%).
△ Less
Submitted 19 January, 2014;
originally announced January 2014.
-
Bayesian rules and stochastic models for high accuracy prediction of solar radiation
Authors:
Cyril Voyant,
C. Darras,
Marc Muselli,
Christophe Paoli,
Marie Laure Nivet,
Philippe Poggi
Abstract:
It is essential to find solar predictive methods to massively insert renewable energies on the electrical distribution grid. The goal of this study is to find the best methodology allowing predicting with high accuracy the hourly global radiation. The knowledge of this quantity is essential for the grid manager or the private PV producer in order to anticipate fluctuations related to clouds occurr…
▽ More
It is essential to find solar predictive methods to massively insert renewable energies on the electrical distribution grid. The goal of this study is to find the best methodology allowing predicting with high accuracy the hourly global radiation. The knowledge of this quantity is essential for the grid manager or the private PV producer in order to anticipate fluctuations related to clouds occurrences and to stabilize the injected PV power. In this paper, we test both methodologies: single and hybrid predictors. In the first class, we include the multi-layer perceptron (MLP), auto-regressive and moving average (ARMA), and persistence models. In the second class, we mix these predictors with Bayesian rules to obtain ad-hoc models selections, and Bayesian averages of outputs related to single models. If MLP and ARMA are equivalent (nRMSE close to 40.5% for the both), this hybridization allows a nRMSE gain upper than 14 percentage points compared to the persistence estimation (nRMSE=37% versus 51%).
△ Less
Submitted 18 September, 2013;
originally announced September 2013.
-
Time series modeling with pruned multi-layer perceptron and 2-stage damped least-squares method
Authors:
Cyril Voyant,
Wani W. Tamas,
Christophe Paoli,
Aurélia Balu,
Marc Muselli,
Marie Laure Nivet,
Gilles Notton
Abstract:
A Multi-Layer Perceptron (MLP) defines a family of artificial neural networks often used in TS modeling and forecasting. Because of its "black box" aspect, many researchers refuse to use it. Moreover, the optimization (often based on the exhaustive approach where "all" configurations are tested) and learning phases of this artificial intelligence tool (often based on the Levenberg-Marquardt algori…
▽ More
A Multi-Layer Perceptron (MLP) defines a family of artificial neural networks often used in TS modeling and forecasting. Because of its "black box" aspect, many researchers refuse to use it. Moreover, the optimization (often based on the exhaustive approach where "all" configurations are tested) and learning phases of this artificial intelligence tool (often based on the Levenberg-Marquardt algorithm; LMA) are weaknesses of this approach (exhaustively and local minima). These two tasks must be repeated depending on the knowledge of each new problem studied, making the process, long, laborious and not systematically robust. In this paper a pruning process is proposed. This method allows, during the training phase, to carry out an inputs selecting method activating (or not) inter-nodes connections in order to verify if forecasting is improved. We propose to use iteratively the popular damped least-squares method to activate inputs and neurons. A first pass is applied to 10% of the learning sample to determine weights significantly different from 0 and delete other. Then a classical batch process based on LMA is used with the new MLP. The validation is done using 25 measured meteorological TS and cross-comparing the prediction results of the classical LMA and the 2-stage LMA.
△ Less
Submitted 8 August, 2013;
originally announced August 2013.
-
Multi-horizon solar radiation forecasting for Mediterranean locations using time series models
Authors:
Cyril Voyant,
Christophe Paoli,
Marc Muselli,
Marie Laure Nivet
Abstract:
Considering the grid manager's point of view, needs in terms of prediction of intermittent energy like the photovoltaic resource can be distinguished according to the considered horizon: following days (d+1, d+2 and d+3), next day by hourly step (h+24), next hour (h+1) and next few minutes (m+5 e.g.). Through this work, we have identified methodologies using time series models for the prediction h…
▽ More
Considering the grid manager's point of view, needs in terms of prediction of intermittent energy like the photovoltaic resource can be distinguished according to the considered horizon: following days (d+1, d+2 and d+3), next day by hourly step (h+24), next hour (h+1) and next few minutes (m+5 e.g.). Through this work, we have identified methodologies using time series models for the prediction horizon of global radiation and photovoltaic power. What we present here is a comparison of different predictors developed and tested to propose a hierarchy. For horizons d+1 and h+1, without advanced ad hoc time series pre-processing (stationarity) we find it is not easy to differentiate between autoregressive moving average (ARMA) and multilayer perceptron (MLP). However we observed that using exogenous variables improves significantly the results for MLP . We have shown that the MLP were more adapted for horizons h+24 and m+5. In summary, our results are complementary and improve the existing prediction techniques with innovative tools: stationarity, numerical weather prediction combination, MLP and ARMA hybridization, multivariate analysis, time index, etc.
△ Less
Submitted 22 July, 2013;
originally announced July 2013.
-
Urban ozone concentration forecasting with artificial neural network in Corsica
Authors:
Wani W. Tamas,
Gilles Notton,
Christophe Paoli,
Cyril Voyant,
Marie Laure Nivet,
Aurélia Balu
Abstract:
Atmospheric pollutants concentration forecasting is an important issue in air quality monitoring. Qualitair Corse, the organization responsible for monitoring air quality in Corsica (France) region, needs to develop a short-term prediction model to lead its mission of information towards the public. Various deterministic models exist for meso-scale or local forecasting, but need powerful large var…
▽ More
Atmospheric pollutants concentration forecasting is an important issue in air quality monitoring. Qualitair Corse, the organization responsible for monitoring air quality in Corsica (France) region, needs to develop a short-term prediction model to lead its mission of information towards the public. Various deterministic models exist for meso-scale or local forecasting, but need powerful large variable sets, a good knowledge of atmospheric processes, and can be inaccurate because of local climatical or geographical particularities, as observed in Corsica, a mountainous island located in a Mediterranean Sea. As a result, we focus in this study on statistical models, and particularly Artificial Neural Networks (ANN) that have shown good results in the prediction of ozone concentration at horizon h+1 with data measured locally. The purpose of this study is to build a predictor to realize predictions of ozone and PM10 at horizon d+1 in Corsica in order to be able to anticipate pollution peak formation and to take appropriated prevention measures. Specific meteorological conditions are known to lead to particular pollution event in Corsica (e.g. Saharan dust event). Therefore, several ANN models will be used, for meteorological conditions clustering and for operational forecasting.
△ Less
Submitted 4 June, 2013;
originally announced June 2013.
-
Hybrid methodology for hourly global radiation forecasting in Mediterranean area
Authors:
Cyril Voyant,
Marc Muselli,
Christophe Paoli,
Marie Laure Nivet
Abstract:
The renewable energies prediction and particularly global radiation forecasting is a challenge studied by a growing number of research teams. This paper proposes an original technique to model the insolation time series based on combining Artificial Neural Network (ANN) and Auto-Regressive and Moving Average (ARMA) model. While ANN by its non-linear nature is effective to predict cloudy days, ARMA…
▽ More
The renewable energies prediction and particularly global radiation forecasting is a challenge studied by a growing number of research teams. This paper proposes an original technique to model the insolation time series based on combining Artificial Neural Network (ANN) and Auto-Regressive and Moving Average (ARMA) model. While ANN by its non-linear nature is effective to predict cloudy days, ARMA techniques are more dedicated to sunny days without cloud occurrences. Thus, three hybrids models are suggested: the first proposes simply to use ARMA for 6 months in spring and summer and to use an optimized ANN for the other part of the year; the second model is equivalent to the first but with a seasonal learning; the last model depends on the error occurred the previous hour. These models were used to forecast the hourly global radiation for five places in Mediterranean area. The forecasting performance was compared among several models: the 3 above mentioned models, the best ANN and ARMA for each location. In the best configuration, the coupling of ANN and ARMA allows an improvement of more than 1%, with a maximum in autumn (3.4%) and a minimum in winter (0.9%) where ANN alone is the best.
△ Less
Submitted 11 November, 2012;
originally announced November 2012.
-
Numerical Weather Prediction (NWP) and hybrid ARMA/ANN model to predict global radiation
Authors:
Cyril Voyant,
Marc Muselli,
Christophe Paoli,
Marie Laure Nivet
Abstract:
We propose in this paper an original technique to predict global radiation using a hybrid ARMA/ANN model and data issued from a numerical weather prediction model (ALADIN). We particularly look at the Multi-Layer Perceptron. After optimizing our architecture with ALADIN and endogenous data previously made stationary and using an innovative pre-input layer selection method, we combined it to an ARM…
▽ More
We propose in this paper an original technique to predict global radiation using a hybrid ARMA/ANN model and data issued from a numerical weather prediction model (ALADIN). We particularly look at the Multi-Layer Perceptron. After optimizing our architecture with ALADIN and endogenous data previously made stationary and using an innovative pre-input layer selection method, we combined it to an ARMA model from a rule based on the analysis of hourly data series. This model has been used to forecast the hourly global radiation for five places in Mediterranean area. Our technique outperforms classical models for all the places. The nRMSE for our hybrid model ANN/ARMA is 14.9% compared to 26.2% for the naïve persistence predictor. Note that in the stand alone ANN case the nRMSE is 18.4%. Finally, in order to discuss the reliability of the forecaster outputs, a complementary study concerning the confidence interval of each prediction is proposed
△ Less
Submitted 8 January, 2012;
originally announced January 2012.
-
Predictability of PV power grid performance on insular sites without weather stations: use of artificial neural networks
Authors:
Cyril Voyant,
Marc Muselli,
Christophe Paoli,
Marie Laure Nivet,
Philippe Poggi,
P. Haurant
Abstract:
The official meteorological network is poor on the island of Corsica: only three sites being about 50 km apart are equipped with pyranometers which enable measurements by hourly and daily step. These sites are Ajaccio (41\degree 55'N and 8\degree 48'E, seaside), Bastia (42\degree 33'N, 9\degree 29'E, seaside) and Corte (42\degree 30'N, 9\degree 15'E average altitude of 486 meters). This lack of…
▽ More
The official meteorological network is poor on the island of Corsica: only three sites being about 50 km apart are equipped with pyranometers which enable measurements by hourly and daily step. These sites are Ajaccio (41\degree 55'N and 8\degree 48'E, seaside), Bastia (42\degree 33'N, 9\degree 29'E, seaside) and Corte (42\degree 30'N, 9\degree 15'E average altitude of 486 meters). This lack of weather station makes difficult the predictability of PV power grid performance. This work intends to study a methodology which can predict global solar irradiation using data available from another location for daily and hourly horizon. In order to achieve this prediction, we have used Artificial Neural Network which is a popular artificial intelligence technique in the forecasting domain. A simulator has been obtained using data available for the station of Ajaccio that is the only station for which we have a lot of data: 16 years from 1972 to 1987. Then we have tested the efficiency of this simulator in two places with different geographical features: Corte, a mountainous region and Bastia, a coastal region. On daily horizon, the relocation has implied fewer errors than a "naïve" prediction method based on the persistence (RMSE=1468 Vs 1383Wh/m^2 to Bastia and 1325 Vs 1213Wh/m^2 to Corte). On hourly case, the results were still satisfactory, and widely better than persistence (RMSE=138.8 Vs 109.3 Wh/m^2 to Bastia and 135.1 Vs 114.7 Wh/m^2 to Corte). The last experiment was to evaluate the accuracy of our simulator on a PV power grid localized at 10 km from the station of Ajaccio. We got errors very suitable (nRMSE=27.9%, RMSE=99.0 W.h) compared to those obtained with the persistence (nRMSE=42.2%, RMSE=149.7 W.h).
△ Less
Submitted 13 January, 2010;
originally announced January 2010.
-
Solar radiation forecasting using ad-hoc time series preprocessing and neural networks
Authors:
Christophe Paoli,
Cyril Voyant,
Marc Muselli,
Marie-Laure Nivet
Abstract:
In this paper, we present an application of neural networks in the renewable energy domain. We have developed a methodology for the daily prediction of global solar radiation on a horizontal surface. We use an ad-hoc time series preprocessing and a Multi-Layer Perceptron (MLP) in order to predict solar radiation at daily horizon. First results are promising with nRMSE < 21% and RMSE < 998 Wh/m2.…
▽ More
In this paper, we present an application of neural networks in the renewable energy domain. We have developed a methodology for the daily prediction of global solar radiation on a horizontal surface. We use an ad-hoc time series preprocessing and a Multi-Layer Perceptron (MLP) in order to predict solar radiation at daily horizon. First results are promising with nRMSE < 21% and RMSE < 998 Wh/m2. Our optimized MLP presents prediction similar to or even better than conventional methods such as ARIMA techniques, Bayesian inference, Markov chains and k-Nearest-Neighbors approximators. Moreover we found that our data preprocessing approach can reduce significantly forecasting errors.
△ Less
Submitted 1 June, 2009;
originally announced June 2009.
-
Predictability of PV power grid performance on insular sites without weather stations: use of artificial neural networks
Authors:
Cyril Voyant,
Marc Muselli,
Christophe Paoli,
Marie Laure Nivet,
Philippe Poggi
Abstract:
The official meteorological network is poor on the island of Corsica: only three sites being about 50 km apart are equipped with pyranometers which enable measurements by hourly and daily step. These sites are Ajaccio (seaside), Bastia (seaside) and Corte (average altitude of 486 meters). This lack of weather station makes difficult the predictability of PV power grid performance. This work inte…
▽ More
The official meteorological network is poor on the island of Corsica: only three sites being about 50 km apart are equipped with pyranometers which enable measurements by hourly and daily step. These sites are Ajaccio (seaside), Bastia (seaside) and Corte (average altitude of 486 meters). This lack of weather station makes difficult the predictability of PV power grid performance. This work intends to study a methodology which can predict global solar irradiation using data available from another location for daily and hourly horizon. In order to achieve this prediction, we have used Artificial Neural Network which is a popular artificial intelligence technique in the forecasting domain. A simulator has been obtained using data available for the station of Ajaccio that is the only station for which we have a lot of data: 16 years from 1972 to 1987. Then we have tested the efficiency of this simulator in two places with different geographical features: Corte, a mountainous region and Bastia, a coastal region. On daily horizon, the relocation has implied fewer errors than a naive prediction method based on the persistence (RMSE=1468 Vs 1383Wh/m2 to Bastia and 1325 Vs 1213Wh/m2 to Corte). On hourly case, the results were still satisfactory, and widely better than persistence (RMSE=138.8 Vs 109.3 Wh/m2 to Bastia and 135.1 Vs 114.7 Wh/m2 to Corte). The last experiment was to evaluate the accuracy of our simulator on a PV power grid localized at 10 km from the station of Ajaccio. We got errors very suitable (nRMSE=27.9%, RMSE=99.0 W.h) compared to those obtained with the persistence (nRMSE=42.2%, RMSE=149.7 W.h).
△ Less
Submitted 21 May, 2009;
originally announced May 2009.
