PD8: Probabilistic Tsunami Forecast (PTF) for early warning and rapid post event assessment | ChEESE - Centre of Excellence for Exascale Supercomputing in the area of the Solid Earth

Pilot specification

Title

PD8: Probabilistic Tsunami Forecast (PTF) for early warning and rapid post event assessment

Contact Person

Stefano Lorito (INGV)

Collaborating institutions

UMA, NGI, CINECA, UniNA, GFZ

Associated Natural hazard

Tsunamis, Earthquakes

Software involved

Tsunami-HySEA (Faster than Real Time Simulations, see PD2); Suite of bash, python, C, GMT codes for workflow execution, pre- and post-processing (https://gitlab.rm.ingv.it/); on the intranet and users need authorization)

Usability

Prototype Software as a Service; Results available for selected use-cases

Intranet

Contact

Twitter

Main objective / mission	This service provides a rapid probabilistic forecast of tsunami inundation, following an earthquake offshore or close to the coast, before it actually occurs or before tsunami observations are available. For near-field tsunami early warning (EW) purposes, the large uncertainty about earthquake location and magnitude, as available in the first minutes, are reflected into forecasting uncertainty. For the purpose of supporting rapid post-disaster intervention, for which more time is available, additional source and even tsunami information in the subsequent phases can be exploited to eventually narrow down the tsunami forecast uncertainty. The service also provides an early estimate of the earthquake parameters with their uncertainty when they are not yet available, as a by-product.
Workflow description	Based on real-time seismic parameters, the workflow either uses pre-calculated tsunami scenarios, or prepares par-files to run large ensembles of numerical simulations with the Tsunami-HySEA Monte Carlo version. It then produces the tsunami forecast combining the ensemble simulation output with source probabilities.
PD configuration	The version based on pre-calculated scenarios works in near-real time in the premises of the CAT-INGV Tsunami Warning Centre and NEAMTWS Tsunami Service Provider. It deals with potentially tsunamigenic earthquakes anywhere in the entire Mediterranean Sea (offshore and inland close to the coast). Order of 10-100 k pre-calculated scenarios were run on a 30 arc-sec grid for 8 hours of simulation. In this configuration, PTF output provides exceedance probabilities for tsunami heights just off the coastline for almost equally spaced points of interest every 20 km along the coasts of the Mediterraean Sea. It was run for example for the recent Ierapetra and Samos-Izmir earthquakes and tsunamis in 2020. Using the pre-calculated scenarios, the computing time can be limited to the order of seconds, which is suitable for early warning purposes. The version based on simulation ensembles to be run from scratch on large enough HPC clusters in urgent computing mode was already tested with: a suite of more than 10 recent events in the Mediterranean Sea, that is all those for which the CAT-INGV issued a tsunami alert message in recent years; the 2003 Zemmouri-Boumerdes Mw 6.8 earthquake and tsunami in the western Mediterranean; the NEAMWave17 Mw 8.5 synthetic scenario; the 2010 Mw 8.8 Maule Chile earthquake and tsunami. Also in this case, order of 10-100 k pre-calculated scenarios were run on a 30 arc-sec grid for 8 hours of simulation (and 30 hours for the Maule event in the Pacific). Also in this configuration, PTF output provides exceedance probabilities for tsunami heights in front of the coastline, but for denser almost equally spaced points of interest every 2 km along the coasts. For the Maule event, the output is also retrieved in correspondence of the deep sea DART sensors that measured the tsunami. The computing time of a single simulation on 1 GPU (V100) is for example ~300 s for 8 hours of tsunami propagation within the entire Mediterranean at 30 arcsec (~ 900 m) spatial resolution, or ~2 hours for 40 hours of propagation within the entire Pacific Ocean at 1 arcmin resolution; the total time to run the whole ensemble depends on the ensemble size and on the number of GPUs available. In this configuration, the PTF is then suitable for rapid post-event assessment but in principle can be exploited also for early warning if enough resources are available.
Tested architectures	Tsunami-HySEA has been tested on many different supercomputers, such as CTE-POWER (BSC), DAVIDE and Marconi100 (CINECA), Piz Daint (CSCS), HPC4 and HPC5 (ENI), in different frameworks, among which is worth noting the Project TSU-CAST - TSUnami ForeCASTing in the PRACE Call 20.
Target TRL	6-8 View TRL chart
Relevant stakeholders	Stakeholders already involved in the development phase are: the CAT-INGV NEAMTWS Tsunami Service Provider, the Italian National Civil Protection Department, the ARISTOTLE-eENHSP multi-hazard scientific partnership. Potential future stakeholders are, for example, the tsunami scientific community and tsunami warning centres.
Achievements up to M41	The Mediterranean scale workflow was tested against several real events, both in the version based on pre-calculated scenarios and the version based on on-the-fly simulations in urgent computing mode The founding elements of the PD8 workflow, to function at a global scale, were successfully implemented and the first tests performed against real events in the Pacific Ocean, including dealing with arrays of simulations corresponding to large ensembles of tsunami scenarios necessary for uncertainty quantification in tsunami forecasting through the Tsunami-HySEA Monte Carlo version In the framework of the PRACE Project TSU-CAST awarded to PD8 on the CINECA Marconi100 supercomputer, seismic and tsunami forecasting are being used for PTF calibration, both in the Mediterranean Sea and in the Pacific Ocean; this is the necessary condition for allowing its transition to the status of an operational tool for tsunami early warning
Related work and further information	Selva, J. et al. Probabilistic Tsunami Forecasting (PTF) for Tsunami Early Warning operations. Geophysical Research Abstracts Vol. 21, EGU2019-17775, 2019. EGU General Assembly 2019, solicited. Lovholt, F. et al. Urgent Tsunami Computing. in 2019 IEEE/ACM HPC for Urgent Decision Making (UrgentHPC) 45–50 (IEEE, 2019). doi:10.1109/UrgentHPC49580.2019.00011.

Exascale outlook

Exascale target (capability/capacity)

In the present version(s) HPC is exploited with capacity runs, since there is the primary need to explore source and tsunami variability according to source uncertainty in real-time. Very large ensembles allow extended uncertainty description using the Monte-Carlo version of Tsunami-HySEA. For this reason, the resolution of the single runs is kept relatively limited to allow many simulations (e.g. each on a different GPU) describing the impact on very large domains depending on their extent (e.g. the Mediterranean Sea or the Pacific Ocean).

However, using parallelism will be considered to locally increase the resolution (e.g. PD2, PD7) with higher capacity runs exploiting also the multi-GPU Tsunami-HySEA features.

Main benefits expected from large increase in compute resources

Increasing the computational resources would allow both deeper uncertainty analysis, both regarding output sensitivity to source details, but also to uncertainty of tsunami numerical modelling.

Large resources may allow obtaining these results in a time compatible with post-event assessment and even early warning requirements.

Moreover, the end-user may benefit from a higher resolution description of the expected tsunami variability, for example during the inundation phase, in turn allowing for better addressing potential consequences.

Main challenges in harnessing large compute resources

An optimization of the workflow is still needed and it is ongoing for more efficient pre- and post-processing and better managing of the data transfer between monitoring centre and HPC centre. Immediate high-availability of failure-proof resources following a potentially tsunamigenic seismic event is probably one challenge. Other challenges are described for PD2 which deals with Tsunami-HySEA that is the HPC engine for PD8.

Specific actions towards exascale at present time

Further than PD2 (HPC code) and PD7 (high-resolution inundation maps) planned developments, a specific interaction for PD8 itself is being triggered with HLRS in the framework of WP3 for the workflow management.

Computational details

	Number of cores / GPUs:	Memory (GB):	Storage (GB) both temporal and permanent:	#files written both temporal and permanent	I/O data traffic per hour during job
*Minimum:**	256 GPUs	16 GB/GPU	80 GB	40k	26 GB
*Average:**	512 GPUs	16 GB/GPU	2 TB	40k	600 GB
*Maximum:**	1024** GPUs	32 GB/GPU	4 TB	40k	1.3 TB

* Min to Max refers to increasing domain size and simulation duration for an application dealing with Rapid Post event assessment performed in a target time of 3 hours with an ensemble size of 20000 scenarios.

** Using the maximum is recommended by the HPC centers. So far, the 1024 number of contemporaneous GPUs are limited by the availability of resources through PRACE, and it would be desirable to increase this number for testing potential scale up towards Exascale.