TOPIO

Focusing on applications of earth system model simulations performed at the University of Hohenheim, TOPIO is investigating the read and write rates for large data volumes on high-performance file systems. The project is also investigating how a significant reduction in the amount of data can be achieved without significant loss of information.

Specifically, two aspects of input/output (I/O) operations will be optimized: the amount of data that is written and the efficiency with which this data is written. The former is being improved using compression methods, the latter using an auto-tuning approach that both hides the complexity of multiple I/O layers from users and is intended to improve I/O performance.

The application used in this project will be the Model for Prediction Across Scales (MPAS), a community earth system model developed by the National Center for Atmospheric Research (NCAR) in the United States. This global numerical weather prediction (NWP) model contains high resolution of a few kilometers on time scales up to a duration of approximately two to three months. Unlike traditional limited-area models using a latitude-longitude grid, MPAS applies an unstructured Voronoi mesh.

In this project MPAS will be applied on a convection permitting (CP) resolution with grid increments on the order of few kilometres at the sub-seasonal time scale (2-3 months). With this model configuration, the data volume can easily exceed 100 TB, especially if ensemble simulations are applied. As it is both time consuming to write this data and expensive to store it, the goal is to reduce data volume considerably while optimizing the write performance of such simulations by applying a variety of existing and newly developed compression algorithms and automatic I/O optimizations. The information loss should not exceed 1 % of information.