Probabilistic tsunami hazard assessment (PTHA) | ChEESE - Centre of Excellence for Exascale Supercomputing in the area of the Solid Earth

Pilot specification

Title

Probabilistic tsunami hazard assessment (PTHA)

Contact Person

Finn Løvholt (NGI)

Collaborating institutions

NGI, INGV, UMA, CINECA

Associated Natural hazard

Tsunamis

Software involved

Tsunami-HySEA, wrapped in a local probabilistic tsunami hazard analysis workflow

Usability

The usability includes both the availability of the workflow and the Tsunami-HySEA software for running the individual tsunami simulations. Due to the complexity of the tsunami hazard analysis, involving tens of thousands of runs, the collaborating institutions will operate the simulations and provide them as a service.

Intranet

Contact

Twitter

Main objective / mission	This Pilot Demonstrator PD7 uses Tier-0 HPC resources to provide Probabilistic Tsunami Hazard Analysis (PTHA), which quantifies the probability of exceeding a specified inundation height, as well as other hazard intensities. PTHA provides scientific guidance for tsunami risk analysis and risk management, including coastal planning and early warning. PTHA aggregates the hazard from many discrete potential scenarios considering their individual probabilities. The objective of the present pilot demonstrator is to provide explicit computation of site-specific PTHA covering a domain of a city scale, say, using a high number of earthquake tsunami source scenarios, combined with high-resolution numerical inundation modelling. The PD7 is linked to applications in the Mediterranean Sea, and uses input data from an existing regional assessment (TSUMAPS-NEAM), to form the basis for the local hazard analysis.
Workflow description	The employed PTHA workflow is shown above. We note that the processes carried out inside the Tier-0 system are shown inside the green box. In brief, the workflow contains the following main steps: (1) Preparation of input data, including user specifications and data retrieval from TSUMAPS-NEAM. This includes high resolution bathymetry and topography data, hazard related thresholds, Tier-0 computational requirements, physics-related input, and data from the previous TSUMAPS-NEAM assessment. (2) HPC workflow for the hazard analysis including high-resolution numerical tsunami simulations in the Tier-0 queue system using a Faster-Than-Real-Time (FTRT) micro-workflow, individual stability checking and removal of spurious results and data reduction algorithms for the individual out fields from the single simulation results. (3) The aggregation of the hazard and visualisation. This is performed outside the Tier-0 system.
PD configuration	Test case conducted for the tsunami hazard in Catania using a total of 32363 earthquake scenarios, using a system of nested grids with 10m-spacing at the finest resolution. A complete Tsunami-HySEA tsunami simulation for Catania with the grid system presented here took approximately 25 minutes on the MARCONI-100 supercomputer at CINECA, meaning that approximately 13600 GPU hours are required to complete the set of scenarios selected. With 1024 simulations able to run in parallel, around 14 hours of clock-time are needed to perform these calculations.
Tested architectures	The software has been tested on four Tier-0 supercomputers: the CTE-POWER cluster in the Barcelona Supercomputing Center (BSC), DAVIDE cluster at CINECA consortium, on Marconi-100, and Piz-Daint at the Swiss National Supercomputer Center. CTE-POWER has 4 Tesla V100 cards per node, while DAVIDE has 4 Tesla P100 in each node and Piz-Daint has only 1 Tesla P100 in each node, while Marconi-100 has 4 V100 GPUs on each node.
Target TRL	5-6
Relevant stakeholders	Relevant stakeholders include, but are not limited to, national and local authorities responsible for managing risks from natural hazards, the insurance and re-insurance industry, owners of industrial installations or critical coastal facilities, and researchers in geoscience and natural hazard mitigation.
Achievements up to M18	The main achievement of PD7 has been to reach a landmark set of high-resolution tsunami hazard maps for Catania using a total of 32363 earthquake scenarios. Moreover, through the new PRACE project TsuHazAP, this source coverage will be greatly expanded, and will likely serve as a benchmark for other similar applications to test needs for source and model resolutions in future PTHA applications in the Mediterranean and elsewhere.
Related work and further information	Several peer reviewed papers reporting on this subject are in the pipeline, and we hope that they will become available in the next few weeks and months.

Exascale outlook

Exascale target (capability/capacity)

We benefit from HPC resources being able to carry out many moderately sized simulations in parallel. At present, identified scalability issues are small, and the ability to scale up is presently limited by the size of the access. Although large Tier-0 resources are available, testing possible limitations would require access to even larger resources, towards Exascale.

Main benefits expected from large increase in compute resources

Larger resources would enable both increasing the complexities we can resolve with the models, as well as performing the assessment much faster. Both are highly beneficial for practical purposes; a local tsunami hazard analysis could be carried out in a matter of a few days, and reveal the sensitivity to a much higher degree of parameter sensitivity than we do today. More resources would also bridge gaps between tsunami hazard analysis (long term) and early warning.

Main challenges in harnessing large compute resources

One of the main challenges is the amount of data produced. Storage and file transfer issues persist. More direct post processing and hazard aggregation procedures would be desirable.

Specific actions towards exascale at present time

The next step is to update the workflow to include source refinement and stability treatment, as well as involving workflow expertise to optimize workflow procedures.

Computational details

	Number of cores / GPUs:	Memory (GB):	Storage (GB) both temporal and permanent:	#files written both temporal and permanent	I/O data traffic per hour during job
Minimum:	4 GPUs	192	2	1000	2 GB
Average:	64 GPUs	768	20	15000	20 GB
Maximum:	1024*	12288	300	200000	300 GB

* Using the maximum is recommended by the HPC centers. So far, the 1024 number of contemporaneous GPUs are limited by the availability of resources through PRACE, and it would be desirable to increase this number for testing potential scale up towards Exascale.