XSHELLS | ChEESE - Centre of Excellence for Exascale Supercomputing in the area of the Solid Earth

Code name	XSHELLS
Developer(s)	Nathanael Schaeffer
Link	https://nschaeff.bitbucket.io/xshells/
Short description	XSHELLS simulates incompressible fluids in a spherical cavity. In addition to the Navier-Stokes equation with an optional Coriolis force, it can also time-step the coupled induction equation for MHD (with imposed magnetic field or in a dynamo regime), as well as the temperature (and concentration) equation in the Boussinesq framework. Based also on a semi-spectral approach combining finite differences in radius and spherical harmonics, semi-implicit second-order time scheme.
Original code level	1
Current code level	1
Pilot(s) involved	PD11
Main results and References	Code audit driven improvements on MPI-3 Shared memory leading to faster time-to-solution and better strong scaling.
Performance results	Comparison of different MPI libraries: Performance using different MPI libraries (Open MPI and Intel MPI library) was checked in order to exploit Intel KNL system performance. Comparing both libraries for a small (NR= 336 radial shells) and a medium testcase (NR= 512 radial shells) similar performance has been found (less than 10% difference) for task/threads decomposition. Scaling and runtime: For a medium test case (NR=512) the version with MPI-3 Shared memory presents overall performance improvements by a factor 1.6 using up to 64 KNL nodes (i.e. 4096 task). No gain using a new preliminary the MPI_ALLTOALL version was found. Communication pattern: The new communication pattern groups all communication into one MPI_ALLTOALL call to make sure all processes have independent work to do. It increases significantly the amount of data to transfer, but reduces the "idle" time. Our tests on KNL show that it does not help yet. In a future work, we must evaluate if it has any interest at all.

XSHELLS