A Framework for Automatic Validation and Application of Lossy Data Compression in Ensemble Data Assimilation
Authors:
Kai Keller,
Hisashi Yashiro,
Mohamed Wahib,
Balazs Gerofi,
Adrian Cristal Kestelman,
Leonardo Bautista-Gomez
Abstract:
Ensemble data assimilation techniques form an indispensable part of numerical weather prediction. As the ensemble size grows and model resolution increases, the amount of required storage becomes a major issue. Data compression schemes may come to the rescue not only for operational weather prediction, but also for weather history archives. In this paper, we present the design and implementation o…
▽ More
Ensemble data assimilation techniques form an indispensable part of numerical weather prediction. As the ensemble size grows and model resolution increases, the amount of required storage becomes a major issue. Data compression schemes may come to the rescue not only for operational weather prediction, but also for weather history archives. In this paper, we present the design and implementation of an easy-to-use framework for evaluating the impact of lossy data compression in large scale ensemble data assimilation. The framework leverages robust statistical qualifiers to determine which compression parameters can be safely applied to the climate variables. Furthermore, our proposal can be used to apply the best parameters during operation, while monitoring data integrity. We perform an exemplary study on the Lorenz96 model to identify viable compression parameters and achieve a 1/3 saving in storage space and an effective speedup of 6% per assimilation cycle, while monitoring the state integrity.
△ Less
Submitted 4 October, 2024;
originally announced October 2024.
Exploring Correlation Patterns in the Ethereum Validator Network
Authors:
Simon Brown,
Leonardo Bautista-Gomez
Abstract:
There have been several studies into measuring the level of decentralization in Ethereum through applying various indices to indicate the relative dominance of entities in different domains in the ecosystem. However, these indices do not capture any correlation between those different entities, that could potentially make them the subject of external coercion, or covert collusion. We propose an in…
▽ More
There have been several studies into measuring the level of decentralization in Ethereum through applying various indices to indicate the relative dominance of entities in different domains in the ecosystem. However, these indices do not capture any correlation between those different entities, that could potentially make them the subject of external coercion, or covert collusion. We propose an index that measures the relative dominance of entities based on the application of correlation factors. We posit that this approach produces a more nuanced and accurate index of decentralization.
△ Less
Submitted 22 March, 2024;
originally announced April 2024.
