PD10: Array-based statistical source detection and restoration and Machine learning from earthquake/volcano slow-earthquakes monitoring | ChEESE - Centre of Excellence for Exascale Supercomputing in the area of the Solid Earth

Pilot specification

Title

PD10: Array-based statistical source detection and restoration and Machine learning from earthquake/volcano slow-earthquakes monitoring

Contact Person

Jean-Pierre Vilotte (IPGP)

Natalia Poita (IPGP)

Collaborating institutions

IPGP, BSC (including Workflows and Distributed Computing Research Department)

Associated Natural hazard

Earthquakes, volcanos, anthropogenic seismicity

Software involved

BackTrackBB: https://github.com/BackTrackBB
PyCOMPSs: https://pypi.org/project/pycompss/

Usability

PyCOMPSs-based BackTrackBB seismic source detection and location code optimized for distributed HPC and Cloud infrastructures.

Generated scientific data product: detailed catalogs of seismic sources.

Intranet

Contact

Twitter

Main objective / mission	Plain language description: PyCOMPSs-based BackTrackBB: use PyCOMPSs framework developed at Barcelona Supercomputing Center (BSC), i.e., the python binding of the COMPSs abstract programming model and runtime, for efficient task-based parallel execution and monitoring on distributed HPC and Cloud infrastructures, together with data movement, of the python-based BackTrackBB data-streaming workflow for automatic detection and location of seismic sources, developed at IPGP, (https://github.com/ BackTrackBB/). To ease the development and deployment of BackTrackBB on different HPC and Cloud environments.
Workflow description	Streaming data intensive analysis. Time-dependent data logistics from edge to centralised HPC and Cloud environments. PyCOMPSs orchestrate task scheduling and data movement within Cloud and HPC environment. HPC & Cloud data ingestion currently require interaction, and should be made automated.
PD configuration	Conducted test cases: 1). Synthetic dataset 7-day continuous data from 100 stations (1 component, 100 sps) 2500 km2 coverage area; idealized homogeneous velocity model. Used for scalability and memory tests. Computing time for 100 station, 1-hour continuous data example ~ 16 min on MareNostrum4 (768 cores). 2). Real-case dataset: 4-year long continuous data from Hi-net seismic stations in Shikoku, Japan (33 stations, 3-components, 100 sps sampling) 57000 km2 coverage area. Target – deep tectonic tremor and low-frequency activity analysis.
Tested architectures	First version of PyCOMPSs-based BackTrackBB implementation has been tested on MareNostrum4 cluster of BSC. Future planned tests on the S-CAPAD platform of IPGP.
Target TRL	4 View TRL chart
Relevant stakeholders	At present time: Research Institutions (IPGP, BSC, ERI), ERC project SEISMAZE. Targetted stakeholders: French Volcanic/Seismological Observatories, ESFRI EPOS.
Achievements up to M41	A summary on the gains obtained within ChEESE First version of PyCOMPs-based BackTrackBB including signal processing optimization (initially serial execution) able to exploit multiple cluster nodes simultaneously and making use of PyCOMPSs task decorators. Scalability tests and memory usage analysis: performed on MareNostrum4 cluster of BSC using different job configurations and case-studies of different dimensions. Scalability test configuration: Data – synthetic dataset: 1-hour long continuous data, 100 sps sampling rate recorded at 100 stations (single component); Architecture – MareNostrum4 cluster of BSC, master worker architecture using 4, 8 and 16 nodes with 48 CPU /node. Memory usage analysis: Data – synthetic datasets with different dimensions (number of stations and grid discretization); Architecture – MareNostrum4 cluster of BSC high memory (7928 MB/core) and standard nodes (1880 MB/core) Ongoing tests for large-scale data sets of continous waveforms using synthetic and real use cases: Synthetic toy model – 7-day, 100 sps sampled, single-component, continuous data from 100 stations. Covered surface area ~ 50 km × 50 km. Real-case data analysis of slow earthquakes activity in southwest Japan – 4-year long seismic recordings sampled at 100 sps from 33 (3-component) Hi-net stations. Covered surface area ~300 km × 190 km.
Related work and further information	Related academic papers: Poiata, N., C., Satriano, J.-P.Vilotte, P., Bernard, and K. Obara (2016). Multi-band array detection and location of seismic sources recorded by dense seismic networks, Geophys. J. Int., 205(3), 1548–1573. Li, L., Tan, J., Schwartz, B., Stanek, F., Poiata, N., Peidong, S., Diekmann, L., L., Eisner, and D., Gajewski (2020). Recent advances and challenges of waveform-based seismic location methods at multiple scales, Reviews of Geophysics, 58, e2019RG000667. Poiata, N., C., J.-P.Vilotte, N., Shapiro, M. Supino and K. Obara (2020). Complexity of Deep Low-Frequency Earthquake Activity Along the Subducting Slab in Western and Central Shikoku, Japan (in preparation). Poiata, N., N. Poiata, J. Conejero, R.M. Badea and J.-P. Vilotte, (2020) BackTrackBB data-streaming workflow for seismic source location using PyCOMPSs parallel computational framework (in preparation). Website related to real use case: http://www-solid.eps.s.u-tokyo.ac.jp/~sloweq/ Website related to BackTrackBB documentation: http://backtrackbb.readthedocs.io/

Exascale outlook

Exascale target (capability/capacity)

Benefit from HPC comes by means of very large parallel simulations (i.e. capability runs).

Main benefits expected from large increase in compute resources

Increase of computational resources allows to obtain faster results and streaming analysis of larger volumes of continuous seismic waveforms, such as longer continuous time series acquired by larger (in term of number of stations) networks. This will lead to more detailed (in terms number and type of events) seismic event catalogues covering longer time-scales and providing a more detailed view of the seismic activity and transient energy release processes in the active regions.

Main challenges in harnessing large compute resources

BackTrackBB is memory intensive application representing main bottleneck. Potential algorithmic changes would be focused on bypassing this problem and improving hybrid task/multithread prallelisation.

In-situ data flow management together with data logistics from edge to centralised environments represent an important issue.

Specific actions towards exascale at present time

PyCOMPSs-based implementation for efficient task-based parallel execution and monitoring.

Extensive testing on different HPC penvironments using various case study data sets.

Computational details

	Number of cores / GPUs:	Memory (GB):	Storage (GB) both temporal and permanent:	#files written both temporal and permanent	I/O data traffic per hour during job
Minimum:	1 serial execution	-	Dependent on study case	No temporary files written Permanent - 1	-
Average:	Depends on the dataset - length of analysed time-series	1880 MB/core (as tested)	10^-1-10⁰TB	10⁴-10⁵	-
Maximum:	Data dependent: Tested - 780 CPUs Expected ~ 25×10⁶	7928 MB/core (as tested)	10¹-10²TB mostly input data; output < 10²MB	Data dependent > 10⁶ permanent files	-