| Main objective / mission |
The aim of the present pilot is to provide robust and very efficient numerical codes for FTRT Tsunami simulations that can be run in massively parallel multi-GPU architectures. |
| Workflow description |
A figure (Figure 1) describing the Workflow is shown at the end of this document.
The Faster-Than-Real-Time (FTRT) prototype for extremely fast and robust tsunami simulations is based upon GPU/multi-GPU (NVIDIA) architectures and is able to use earthquake information from different locations and with heterogeneous content (full Okada parameter set, hypocenter and magnitude plus Wells and Coppersmith (1994). Using these inhomogeneous inputs, and according to the FTRT workflow (see Fig. 1), tsunami computations are launched for a single scenario or a set of related scenarios for the same event. Basically, the automated retrieval of earthquake information is sent to the system and on-the-fly simulations are automatically launched. Therefore, several scenarios are computed at the same time. As updated information about the source is provided, new simulations should be launched. As output, several options are available tailored to the end-user needs, selecting among: sea surface height and its maximum, simulated isochrones and arrival times to the coastal areas, estimated tsunami coastal wave height, times series at Points of Interest (POIs) and oceanographic sensors.
A first successful implementation has been done for the Emergency Response Coordination Centre (ERCC), a service provided by the ARISTOTLE-ENHSP Project (http://aristotle.ingv.it/tiki-index.php). The system implemented for ARISTOTLE follows the general workflow presented in Figure 1 . Currently, in this system, a single source is used to assess the hazard and the computational grids are predefined. The computed wall-clock time is provided for each experiment and the outputs of the simulation are maximum water height and arrival times on the whole domain, and water height time-series on a set of selected POIs, predefined for each domain.
A library of Python codes are used to generate the input data required to run HySEA codes and to extract the topo-bathymetric data and construct the grids used by HySEA codes. |
| PD configuration |
PD consists of several predefined computational areas (Pacific, regional Japan, Western Mediterranean, the whole Mediterranean, …) where a complete set-up is made available. This allows performing any requested scenario in the selected region simply changing the input seismic source. Other configurations for massive predetermined simulations are also available. |
| Tested architectures |
Tsunami-HySEA has been tested on CTE-POWER (BSC), DAVIDE and Marconi100 (CINECA) and Piz Daint (CSCS) supercomputers. |
| Target TRL |
Currently we have already reached the highest TRL levels (8-9) as the PD is being used in complete and qualified systems, and proven in operational environments. This has been achieved in the Aristotle-ENHSP project, providing the Tsunami Service and also at the CAT (Centro Allerta Tsunami) at INGV.
View TRL chart
|
| Relevant stakeholders |
-
National Tsunami Early Warning Systems. Currently INGV (Italy), IGN (Spain), PMEL/NOAA (US), future ones SHOA (Chile), SINAMOT (Costa Rica).
-
Tsunami Risk Assessment. Present time: Junta de Andalucía (Spain), Seismic Network (Puerto Rico, US), Italian Civil Protection, UNESCO projects in the Caribbean, PMEL/NOAA (USA), …
-
Aristotle e-ENHSP project (ERCC)
-
Insurance companies. Currently CCS (Spain).
-
IUB members: EPOS, Italian Civil Protection, IGN, Aristotle-ENHSP, …
-
Projects: eFlows4HPC (H2020), TSU-CAST, TsuHazAP, UMA-CEIATECH-05, AGITHAR COST Action, AECT-2020--1-0009, GEOMEP Tsunamis, …
-
Copernicus Activations (INDRA, Spain)
|
| Achievements up to M41 |
Improving HySEA codes in CHEESE:
-
We have improved the load balancing algorithm. In particular we have added support for heterogeneous GPUs in the load balancing algorithm by assigning a numerical weight to each GPU.
-
We have developed a new algorithm for the nested meshes processing based on the current state values and we have implemented the activation of the nested meshes processing when a movement of the water is detected in their area.
-
Implemented asynchronous file writing by creating an additional thread for each MPI process using C++11 threads (see Table 1 at the end of this document).
-
Added the possibility of resuming a stored simulation.
-
Added sponge layers for a better processing of the border conditions, in order to avoid possible numerical instabilities in the borders of the domain.
-
Implemented asynchronous CPU-GPU memory transfers.
-
We have dramatically reduced the size of the output files compressing the data using the algorithm described in Tolkova (2008) and saving most of the data in single precision files.
-
A new version of Tsunami-HySEA has been developed to run simultaneous simulations on the same domain attending the requirements of PD7 and PD8, by executing one simulation on each GPU. This new version is able to use up to 1024 GPUs simultaneously with a very good weak scaling (losing around 3% of efficiency).
With these improvements, we obtain around 30% of reduction on the computational time with respect to the previous version of the codes (see Figure 2 at the end of this document)
The codes have been tested on CTE-POWER (BSC), DAVIDE and Marconi100 (CINECA) and Piz Daint (CSCS) supercomputers. |
| Related work and further information |
HySEA codes web page: https://edanya.uma.es/hysea/
References: https://edanya.uma.es/hysea/index.php/references
Pieces of news in the media (in Spanish):
https://www.uma.es/edanya/cms/menu/news-in-the-media/ |
