TY - JOUR
T1 - Large-scale flow simulations using lattice Boltzmann method with AMR following free-surface on multiple GPUs
AU - Watanabe, Seiya
AU - Aoki, Takayuki
N1 - Funding Information:
This research was partly supported by KAKENHI, Grant-in-Aid for Scientific Research (S) JP26220002 and JP19H05613 from Japan Society for the Promotion of Science (JSPS) and ?Joint Usage/Research Center for Interdisciplinary Large-scale Information Infrastructures (JHPCN), Japan? jh180034, jh180035 and ?High Performance Computing Infrastructure (HPCI), Japan? hp190130. The first author was supported by Grant-in-Aid for JSPS Research Fellow JP17J09945 from Japan Society for the Promotion of Science (JSPS) in Japan. The authors thank professor Changhong Hu and assistant professor Makoto Sueyoshi of Kyushu University for the experiment of the dam breaking problem. The numerical calculations were carried out using the computing resources of the TSUBAME3.0 supercomputer at Tokyo Institute of Technology and the computing resources of the ITO supercomputer at Research Institute for Information Technology, Kyushu University.
Publisher Copyright:
© 2021 Elsevier B.V.
PY - 2021/7
Y1 - 2021/7
N2 - Free-surface flow simulations require high-resolution grids to capture phenomena at the interface as well as a long computational time. In this paper, we propose a numerical method for realizing large-scale free-surface flow simulations using the lattice Boltzmann method and multiple GPUs. By introducing the adaptive mesh refinement (AMR) method, which adapts high-resolution grids to free surfaces, to the lattice Boltzmann method, the number of lattice points can be greatly reduced. In the calculation of the AMR method, the spatial distribution of a computational load changes with time; therefore, the number of lattice points assigned to each GPU is kept equal by dynamic domain partitioning using a space-filling curve. We measured the weak scalability of our AMR code on the TSUBAME3.0 supercomputer at the Tokyo Institute of Technology. By hiding GPU–GPU communication overheads by the overlapping method, the performance increased 1.29 times that of the naïve implementation, and we achieved the fairly high performance of 14,570 MLUPS using 256 GPUs. We demonstrate large-scale simulations for the dam breaking problem and show a reduction in computational cost with the AMR method.
AB - Free-surface flow simulations require high-resolution grids to capture phenomena at the interface as well as a long computational time. In this paper, we propose a numerical method for realizing large-scale free-surface flow simulations using the lattice Boltzmann method and multiple GPUs. By introducing the adaptive mesh refinement (AMR) method, which adapts high-resolution grids to free surfaces, to the lattice Boltzmann method, the number of lattice points can be greatly reduced. In the calculation of the AMR method, the spatial distribution of a computational load changes with time; therefore, the number of lattice points assigned to each GPU is kept equal by dynamic domain partitioning using a space-filling curve. We measured the weak scalability of our AMR code on the TSUBAME3.0 supercomputer at the Tokyo Institute of Technology. By hiding GPU–GPU communication overheads by the overlapping method, the performance increased 1.29 times that of the naïve implementation, and we achieved the fairly high performance of 14,570 MLUPS using 256 GPUs. We demonstrate large-scale simulations for the dam breaking problem and show a reduction in computational cost with the AMR method.
UR - http://www.scopus.com/inward/record.url?scp=85102880235&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85102880235&partnerID=8YFLogxK
U2 - 10.1016/j.cpc.2021.107871
DO - 10.1016/j.cpc.2021.107871
M3 - Article
AN - SCOPUS:85102880235
VL - 264
JO - Computer Physics Communications
JF - Computer Physics Communications
SN - 0010-4655
M1 - 107871
ER -