PD12: High-resolution volcanic ash dispersal forecast | ChEESE - Centre of Excellence for Exascale Supercomputing in the area of the Solid Earth

Pilot specification

Title

PD12: High-resolution volcanic ash dispersal forecast

Contact Person

Arnau Folch, Barcelona Supercomputing Center (BSC)

Collaborating institutions

ChEESE partners: BSC, INGV, IMO

ChEESE IUB: Italian Civil Protection, Icelandic Civil Protection, Mitiga Solutions

Associated Natural hazard

Volcanoes

Software involved

FALL3D code (version 8.1; https://gitlab.com/fall3d-distribution )

Combined with a simple workflow (see below)

Usability

Workflow intended as a Software as a Service

Intranet

Contact

Twitter

Main objective / mission	Volcanic ash cloud forecasts are performed shortly before or during an eruption in order to predict expected fallout rates in the next hours/days and/or to prevent aircraft encounters with volcanic clouds. The quantitative forecasts are one of the most important decision tools for flight cancellations and airplane re-routings avoiding contaminated airspace areas. However, an important gap exists between current operational products and the actual requirements from the aviation industry and related stakeholders in terms of model resolution, frequency of forecasts, and quantification of airborne ash concentration. This PD has implemented an ensemble-based data assimilation system (workflow) combining the FALL3D dispersal model with high-resolution geostationary satellite retrievals in order to generate realistic high-resolution forecasts.
Workflow description	PD workflow includes the following components: Downloading and pre-processing of required meteorological data. Downloading of raw satellite data and the production of cloud mass quantitative retrievals (e.g. SEVIRI retrievals at 0.1º resolution, 1-hour frequency). The ensemble forecast execution using the FALL3D model (i.e. the HPC component of the workflow) The post-processing step to generate the final products and figures No WMS available yet (work in progress)
PD configuration	Test case conducted for the 2010 Eyjafjallajökull eruption. Domain covering Europe at 4 km resolution with 70 vertical model levels and 30 ensemble members and a maximum of 6-hourly data assimilation cycles using LETKF filter. Hourly model results with a forecast window of T0+96h. Computing time on Irene-rome (7680 cores): 40 min.
Tested architectures	Mare-Nostrum-IV (BSC) Irene-rome (CEA/TGCC) Irene-skylake (CEA/TGCC)
Target TRL	5 (i.e. individual components of the prototype service demonstrated in relevant environments). View TRL chart
Relevant stakeholders	If applied over a continental scale, a 24/7 early warning system and forecast can be addressed to different stakeholders of the aviation industry, e.g. airlines, Aeronautic Service Providers (ANSPs), airports, local and regional air traffic controllers, etc. If applied on a local/regional scale, a 24/7 ash fallout forecast system, e.g. addressed to civil protection agencies, decision makers and humanitarian sectors. Currently interested IUB members include Mitiga Solutions, ARISTOTLE, and the Icelandic and Italian Civil Protection (ICP)
Achievements up to M41	Implementation of ensemble forecasts in FALL3D to run different ensemble members (model realizations) as a single model run A new workflow component has been developed to retrieve ash (and SO2) cloud column mass from last-generation satellite instrumentation observations A new satellite data assimilation module based on the Parallel Data Assimilation Framework (PDAF; http://pdaf.awi.de/trac/wiki ) has been implemented
Related work and further information	Mingari, L., Folch, A., Prata, A. T., Pardini, F., Macedonio, G., and Costa, A.: Data assimilation of volcanic aerosol observations using FALL3D+PDAF, Atmos. Chem. Phys., 22, 1773–1792, https://doi.org/10.5194/acp-22-1773-2022, 2022. Folch A, Mingari L and Prata AT (2022) Ensemble-Based Forecast of Volcanic Clouds Using FALL3D-8.1. Front. Earth Sci. 9:741841. https://doi.org/10.3389/feart.2021.741841 Prata, A. T., Mingari, L., Folch, A., Macedonio, G., and Costa, A.: FALL3D-8.0: a computational model for atmospheric transport and deposition of particles, aerosols and radionuclides – Part 2: Model validation, Geosci. Model Dev., 14, 409–436, https://doi.org/10.5194/gmd-14-409-2021, 2021. Folch, A., Mingari, L., Gutierrez, N., Hanzich, M., Costa, A., Macedonio, G., FALL3D-8.0: a computational model for atmospheric transport and deposition of particles and aerosols. Part I: model physics and numerics, geoscientific model development, https://doi.org/10.5194/gmd-13-1431-2020, 2020. Osores, S., Ruiz, J., Folch, A., Collini, E., Volcanic ash forecast using ensemble-based data assimilation: the Ensemble Transform Kalman Filter coupled with FALL3D-7.2 model (ETKF-FALL3D, version 1.0), Geoscientific Model Development, https://doi.org/10.5194/gmd-2019-95, 2019.

Exascale outlook

Exascale target (capability/capacity)

Capability. We are interested in time-to-solution, i.e. to fulfill time-constraints in an urgent computing environment (walltime limit of 30-60 min)

Main benefits expected from large increase in compute resources

Increase in resources benefits in two ways. First by increasing the model resolution up to ~1 km grid size (hourly to 15 min output). Second by allowing a larger ensemble size, which permits exploring better the uncertainty range in the Eruption Source Parameters (ESP) and in the underlying meteorological driver.

Main challenges in harnessing large compute resources

FALL3D has been proven to have a good strong scalability (above 90 % of parallel efficiency) up to several thousands of processors (Folch et al., 2020). As the ensemble forecasting task is embarrassingly parallel, major constraints on computing time come from the analysis step.

Specific actions towards exascale at present time

A version of the FALL3D code for accelerators (open ACC pragmas) has been implemented and will be released in v8.2

Computational details

	Number of cores / GPUs:	Memory (GB):	Storage (GB) both temporal and permanent:	#files written both temporal and permanent	I/O data traffic per hour during job
Minimum:	720 cores	8.5 GiB	5.5 GB	300 files
Average:
Maximum:	7680 cores	34 GiB	52 GB	300 files