Hyperparameter Optimisation in Deep Learning from Ensemble Methods: Applications to Proton Structure
Authors:
Juan Cruz-Martinez,
Aaron Jansen,
Gijs van Oord,
Tanjona R. Rabemananjara,
Carlos M. R. Rocha,
Juan Rojo,
Roy Stegeman
Abstract:
Deep learning models are defined in terms of a large number of hyperparameters, such as network architectures and optimiser settings. These hyperparameters must be determined separately from the model parameters such as network weights, and are often fixed by ad-hoc methods or by manual inspection of the results. An algorithmic, objective determination of hyperparameters demands the introduction o…
▽ More
Deep learning models are defined in terms of a large number of hyperparameters, such as network architectures and optimiser settings. These hyperparameters must be determined separately from the model parameters such as network weights, and are often fixed by ad-hoc methods or by manual inspection of the results. An algorithmic, objective determination of hyperparameters demands the introduction of dedicated target metrics, different from those adopted for the model training. Here we present a new approach to the automated determination of hyperparameters in deep learning models based on statistical estimators constructed from an ensemble of models sampling the underlying probability distribution in model space. This strategy requires the simultaneous parallel training of up to several hundreds of models and can be effectively implemented by deploying hardware accelerators such as GPUs. As a proof-of-concept, we apply this method to the determination of the partonic substructure of the proton within the NNPDF framework and demonstrate the robustness of the resultant model uncertainty estimates. The new GPU-optimised NNPDF code results in a speed-up of up to two orders of magnitude, a stabilisation of the memory requirements, and a reduction in energy consumption of up to 90% as compared to sequential CPU-based model training. While focusing on proton structure, our method is fully general and is applicable to any deep learning problem relying on hyperparameter optimisation for an ensemble of models.
△ Less
Submitted 21 October, 2024;
originally announced October 2024.
Reconciling spectroscopy with dynamics in global potential energy surfaces: the case of the astrophysically relevant SiC$_{2}$
Authors:
C. M. R. Rocha,
H. Linnartz,
A. J. C. Varandas
Abstract:
SiC$_2$ is a fascinating molecule due to its unusual bonding and astrophysical importance. In this work, we report the first global potential energy surface (PES) for ground-state SiC$_2$ using the combined-hyperbolic-inverse-power-representation (CHIPR) method and accurate ab initio energies. The calibration grid data is obtained via a general dual-level protocol developed afresh herein that enta…
▽ More
SiC$_2$ is a fascinating molecule due to its unusual bonding and astrophysical importance. In this work, we report the first global potential energy surface (PES) for ground-state SiC$_2$ using the combined-hyperbolic-inverse-power-representation (CHIPR) method and accurate ab initio energies. The calibration grid data is obtained via a general dual-level protocol developed afresh herein that entails both coupled-cluster and multireference configuration interaction energies jointly extrapolated to the complete basis set limit. Such an approach is specially devised to recover much of the spectroscopy from the PES, while still permitting a proper fragmentation of the system to allow for reaction dynamics studies. Besides describing accurately the valence strongly-bound region that includes both the cyclic global minimum and isomerization barriers, the final analytic PES form is shown to properly reproduce dissociation energies, diatomic potentials, and long-range interactions at all asymptotic channels, in addition to naturally reflect the correct permutational symmetry of the potential. Bound vibrational state calculations have been carried out, unveiling an excellent match of the available experimental data on $c$-$\mathrm{SiC}_{2}(^{1}A_1)$. To further exploit the global nature of the PES, exploratory quasi-classical trajectory calculations for the endothermic $\mathrm{C_{2}\!+\!Si}\rightarrow\mathrm{SiC\!+\!C}$ reaction are also performed, yielding thermalized rate coefficients for temperatures up to $5000$ K. The results hint for the prominence of this reaction in the innermost layers of the circumstellar envelopes around carbon-rich stars, thence conceivably playing therein a key contribution to the gas-phase formation of SiC, and eventually, solid SiC dust.
△ Less
Submitted 10 August, 2022;
originally announced August 2022.
