This paper presents the results of comparing a Lagrangian vortex method with a trusted spectral method for the simulation of isotropic fluid turbulence. The numerical engine of the vorticity-based fluid solver is a massively parallel fast multipole method (FMM) running on GPU hardware using CUDA. Thus, there are several aspects to the validation: the particle method itself, the fast summation via FMM, and the use of GPUs in this application.
There have been insufficient efforts so far to fully validate the vortex method in direct numerical simulations of turbulent flows, which has left ample room for skepticism. The benchmark of decaying, homogeneous isotropic turbulence is an ideal test for quantitative validation of fluid solvers, with the simplest possible geometry (a periodic cube) and many years of experience using turbulence statistics in its analysis. It is not the best problem for showcasing a vortex particle method, which shines in wake flows and other flows dominated by vorticity, but it allows quantitative validation. We use a Taylor microscale Reynolds number of 50 and 100 in a 256^3 mesh and look at various turbulence statistics, including high-order moments of velocity derivatives. A parametric study looks at the effect of the order of series expansion in the FMM, the frequency of particle reinitialization in the vortex method, the overlap ratio of particles and the time step. At this resolution, the vortex method matches quantitatively the spectral method at Re=50, but some deviations are appreciable in the velocity skewness and flatness for Re=100.
The vortex method application uses a parallel FMM code, called exaFMM, that runs on GPU hardware using CUDA, while the reference spectral code (developed and used at the Center for Turbulence Research in Stanford) runs on CPU only. Results indicate that, for this application, the spectral method is an order of magnitude faster than the vortex method when using a single GPU for the FMM and six CPU cores for the FFT. The entire code that was used to obtain the present results is available under the MIT license from: https://bitbucket.org/exafmm/
In another publication (coming soon), authors focus on the performance of the FMM-based vortex method on massively parallel systems, compared to the spectral method. That work included simulations of isotropic turbulence on up to 4096 GPUs, with a 4096^3 problem size and exceeding 1 petaflop/s of sustained performance. Preliminary results are depicted on figure above. THis data is available at figshare.
Rio Yokota, L.A. Barba, FMM-based vortex method for simulation of isotropic turbulence on GPUs, compared with a spectral method, Computers & Fluids, Available online 13 August 2012. [DOI: 10.1016/j.compfluid.2012.08.002].
[Source: Professor Lorena Barba, Boston University]