FALL3D | ChEESE - Centre of Excellence for Exascale Supercomputing in the area of the Solid Earth

Code name	FALL3D
Developer(s)	Arnau Folch, Leonardo Mingari, Natalia Gutierrez, Antonio Costa and Giovanni Macedonio Contact person: Arnau Folch
Link	https://gitlab.com/fall3d-distribution
Short description	Open-source off-line Eulerian model for atmospheric passive transport and deposition based on the Advection-Diffusion-Sedimentation (ADS) equation. In FALL3D-8.0 (Folch et al., 2020; Prata et al., 2020), the ADS equation has been extended to handle passive transport of other substances different from tephra (aerosols, particles and radionuclides)
Original code level	3
Current code level	2
Pilot(s) involved	PD3, PD6, PD12
Co-design	Indirectly. The Mini-app “MiniAero” is an explicit (using RK4) unstructured finite volume code that solves the compressible Navier-Stokes equations, applicable to FALL3D. Both inviscid and viscous terms are included. The viscous terms can be optionally included or excluded.
Main results and References	MAIN RESULTS: New code release (8.0) and ensemble-based forecasts with parallel data assimilation implemented (8.1 release). Better communication/computation ratio by refactoring the original (7.3) code version. The speedup increases with the number of cores up to a factor of 4.3x. GPU porting: migrated solver kernels to GPUs using OpenACC directives; the improvement is almost 60% comparing GPU version with CPU version. Vectorization: some directives implemented to enforce vectorization for loops that the compiler does not auto-vectorize. Speedup up to 1.2 achieved. Parallel I/O: A parallel I/O strategy for NetCDF files implemented. The performance, up to a factor 2x, is also improved. REFERENCES: Folch et al., 2020 Prata et al., 2020
Performance results	Strong scalability tests up to 10000 cores (irene-rome and MN-4) with parallel efficiency above 80%.

Ensemble probability of exceedance considering a threshold of 0.2 g/m2 for ash column mass for the 2010 Eyjafjallajökull event. Results before and after the assimilation step (16 April 2010 at 00:00 UTC). Simulation at 4km grid resolution (50M grid points) and 30 ensemble members.