10-4 Massively Parallel Techniques for Realizing 105 Times Higher Spatio-Temporal Resolutions

－The K Computer Sheds Light on Multi-Scale Turbulence in Fusion Plasma－

Fig.10-9　Optimized inter-node communication on 3D torus network

Fig.10-9　Optimized inter-node communication on 3D torus network

The 5D phase space in fusion plasma turbulence is split into small segments, which are allocated on a 3D torus network so as to minimize the required inter-node communications (transpose, point-to-point, and reduction). Thanks to this segmented process mapping, the communication cost is significantly reduced.

 

Fig.10-10　Electrostatic potential fluctuations in multi-scale fusion plasma turbulence

Fig.10-10　Electrostatic potential fluctuations in multi-scale fusion plasma turbulence

We realize multi-scale turbulence simulations that resolve both electron-scale (shown in the magnified view) and ion-scale fluctuations (shown in overall view) consistently, and enable analysis of their cross-scale interactions.

The analysis of turbulent transport is one of the central areas in magnetic fusion research, as the transport caused by electromagnetic turbulent fluctuations has significant impacts on the confinements of fusion fuel in the form of plasma. Numerical simulations of plasma turbulence require significant computation to solve the time evolution of the plasma distributions in a 5-dimensional (5D; three spatial and two velocity coordinates) phase space.

Most of the previous studies have treated the electron-scale turbulence (wavelengths ∼0.5 mm, frequencies ∼400 kHz) and the ion-scale turbulence (wavelengths ∼20 mm, frequencies ∼10 kHz) individually by assuming scale separation. The assumption, however, is not obvious. Hence, it is an important issue to reveal whether there are cross-scale interactions or not. Multi-scale simulations resolving their wavelengths and frequencies require 105 times higher resolutions than those in single-scale simulations. To realize their large computational requirements, we had to wait for the appearance of petascale supercomputers, such as the K computer (K).

K is a distributed parallel computer, which connects a number of computation nodes with an inter-node network. Therefore, inter-node communications often become a bottleneck for performance. Investigating various types of inter-node communications in the 5D phase space problem, we have developed an optimized communication algorithm on K. The key idea is the segmented mapping of the communication processes, so that communications are performed locally and independently on a 3D torus network (Fig.10-9). This significantly reduces the communication cost by ∼60%. Thanks to the optimizations, we have achieved 0.78 PFLOPS using ∼600000 cores on K, and realized multi-scale fusion plasma turbulence simulations (Fig.10-10). The simulation result reveals the existence of cross-scale interactions between the electron and ion scales and the change of the turbulent spectrum from those in single-scale simulations. These achievements were internationally appreciated with the Best Poster Award in International Conference for High Performance Computing, Networking, Storage and Analysis: SC13 among other awards.

The present study was supported by HPCI Strategic Program Field 4 and G8 Research Council Initiative NuFuSE.

Reference
Maeyama, S., Idomura, Y. et al., Computation-Communication Overlap Techniques for Parallel Spectral Calculations in Gyrokinetic Vlasov Simulations, Plasma and Fusion Research, vol.8, 2013, p.1403150-1-1403150-8.

 |