MTech(res) Parallel Particle Tracking for Turbulent Round Jets

Particle tracking in turbulent jets is essential for an accurate Lagrangian understanding of jet flows. While parallel algorithms and numerical schemes to compute flow fields can be scaled to a large number of computing processes, the same is not necessarily true for particle tracking. This thesis presents an analysis of the parallel efficiency of particle tracking in the specific context of round turbulent jets.
A particle tracking module was developed and integrated with Xcompact3d – a well-established flow solver with strong and weak scaling up to 100,000 processes. The module uses trilinear velocity interpolation scheme and a first-order Euler integration scheme with sub-stepping for particle advancement. Velocity data is exchanged via a halo API provided by 2DECOMP&FFT – the domain decomposition library that Xcompact3d builds upon. Particle data is exchanged via MPI collectives. Additional exchange algorithms that aim to avoid the scalability limitations of MPI collectives are available to the user. However, the choice of algorithms for data exchange is often constrained by flow and simulation parameters.

The module is validated using an analytical velocity field and subsequently employed for DNS and LES of turbulent jets with particles. Particle concentration profiles are used to assess accuracy. The results demonstrate the capability of the integrated code to accurately track particles over multiple simulation runs. The particle numbers used for these simulations are comparable or higher than the number of grid points as opposed to most studies in the literature where particle numbers are much lower than the number of grid points.

A detailed wall-time and particle-load analysis is done to assess the parallel efficiency and identify performance limitations. Particle tracking and exchange are the largest contributors to simulation time after flow field computation. The wall-time for particle tracking increases with the number of particles in the domain. A large difference is observed between the average wall-time and maximum wall-time across computing processes which is attributed to particle load imbalance. In contrast, the particle exchange wall-time across computing processes is similar because of the MPI collectives being used for particle data exchange. However, the number of particles, and consequently, the size of data being exchanged, is a small fraction of the number of particles being tracked. A particle load-analysis reveals considerable variation in the number of particles being tracked by individual processes, with the maximum difference in particle count amongst processes being as high as three orders of magnitude.

The results obtained highlight the challenges in implementing particle tracking at scale. While strategies to mitigate the impact of non-scalable collectives for particle exchange exist, algorithms to mitigate the severe particle-load imbalance in the context of turbulent jets have not been adequately studied and are urgently required, which is identified as the key area for future work.

Speaker : Nikhil Jayswal

Research Supervisor: Prof. Joseph Mathew