Intel® Parallel Computing Center
- Honggao Liu, HPRC Director, TAMU
- Zhi Shang, HPRC Post-Doc, TAMU
TAMU's interdisciplinary High Performance Research Computing (HPRC) has a mission to infuse computational technologies into the research and creative activities of every academic discipline. The Intel PCC program at HPRC led by PI Liu will develop open-source software focusing on simulation of flows through micropores, such as those found in rocks involved in oil and gas extraction, by extending OpenFOAM, a popular open-source simulation software. Fig. 1 shows the digital micropores in the rocks in oil reservoir. The primary focus is to modernize OpenFOAM to increase parallelism and scalability through optimizations that leverage cores, caches, threads, and vector capabilities of Intel Xeon Phi processors. For achieving the best speedup using Intel Xeon Phi processors, we are interested in increasing the parallelism by adding OpenMP where applicable, refactoring code to allow compilers to vectorize loops, and identifying candidate arrays for the high-bandwidth, on-package memory to making OpenFOAM run better on Knights Landing processors.
Figure 1. Structure of micropores in oil reservoir.
Gas, oil, water and sand particles all normally exist in a reservoir. The accurate information on the location of oil within porous media (such as rock and sand) is needed to help determine where to drill and to evaluate existing oil reservoirs. It is also important to know where extensive sand deposits exist within porous media. Therefore, there are lots of data processing and physics computing in analyzing the situation of the reservoir. With new methods available to address complex physical phenomena, and advances in powerful computing platforms, the ability to model fundamental flow physics at high resolution becomes both essential and possible. The MP-PIC (multiphase particle-in-cell) method was employed as the DPM (discrete particle modeling) model to perform the multiphase flow simulation in the porous media. Fig. 2 shows the distribution of the sand particles with the oil flows inside the porous media of the reservoir.
Figure 2. Distribution of sand particles with oil flows.
The multiphase flow is fundamental to many engineering and environmental processes related not only to oil & gas but also to chemical processing, energy, and geophysics. Simulation challenges are magnified by the complicated flow phenomena with interactions between different phases. Especially when the simulated domain is approaching industrial scale, the large number of grid plus the complicated flow phenomena will block practical application by traditional numerical simulations. Partly reason is due to current community codes, however, do not exploit HPC capabilities to the fullest and lack fully coupled physics. The Intel PCC project at HPRC will undertake code performance scaling, profiling, and optimization for OpenFOAM on advanced HPC platforms, and develop the multiphysics coupled simulation algorithm suited for both fundamental physics needs as well as efficient usage of HPC resources. The research results will serve as the development methods for extending the current OpenFOAM programming frameworks through incorporation of modules using Intel Xeon Phi coprocessors and processors. Code modernization for scientific and industrial research is critical to advancing the pace of discovery and innovation. Modernizing OpenFOAM codes for Intel architecture will have broad and lasting impact on the community for years to come. Fig. 3 shows the speed up of the large scale CFD simulations (51 million mesh cells with 40 million sand particles) on the oil reservoir using coupled DPM with CFD with OpenFOAM accelerated by Intel Xeon Phi Coprocessors using MPI, OpenMP and vectorization parallel programming. In Fig. 3, 1 node has 20 processors and 1 Xeon Phi Coprocessor has 61 cores.
Figure 3. Speed up of the large scale CFD simulations accelerated by Intel Phi Coprocessors.
- Zhi Shang, Honggao Liu, James A. Lupo. Evaluation of hybrid MPI-OpenMP on discrete particle modeling for large scale parallel computing with OpenFOAM. Asian Journal of Mathematics and Computer Research, available (in press) at http://www.ikpress.org/articles-press/44.
- Zhi Shang, Honggao Liu, James A. Lupo.
Effect of Solid Particles in Porous Media on Permeability and High Performance Coputing at Intel Xeon Phi Coprocessors with OpenFOAM.
International Journal of Modeling, Simulation, and
Scientific Computing, submitted.
High Performance Computing at Intel Xeon Phi Coprocessor for Discrete Particle Model of OpenFOAM.
Society of Exploration Geophysicists (SEG),
International Explosion and 85th Annual Meeting, New Orleans, Louisiana, October 18 - 23, 2015, USA. (http://www.hpcsociety.org/seg2015-abstracts#shang) (presentation)
Zhi Shang, Honggao Liu, Krishnaswamy Nandakumar, Mayank Tyagi, James A. Lupo, Karsten Thompson.
Discrete Particle Model for Porous Media Flow using OpenFOAM at Intel Xeon Phi Coprocessor.
American Physical Society, Division of Fluid Dynamics 68th Annual Meeting, Boston, Massachusetts, November 22 - 24, 2015, USA. (http://meetings.aps.org/Meeting/DFD15/Session/M7.9) (presentation)
Zhi Shang, Mayank Tyagi, James A. Lupo, Honggao Liu, Krishnaswamy Nandakumar, Karsten Thompson.
Large Scale CFD Simulations of Particulate Flows in Porous Media. 27th Supercomputing Conference (SC15), Austin, Texas, November 15 - 20, 2015, USA. (http://sc15.supercomputing.org) (poster)
Last modified: 7 Jun 2016