OpenFOAM Performance with Cornelis Networks OPX

This article presents a technical report on the performance and scalability of OpenFOAM when utilized with Cornelis Networks' Omni-Path Express (OPX) fabric on AMD EPYC 7003-series processors. The report compares the performance of OpenFOAM benchmarks on clusters connected by Cornelis OPX and NVIDIA InfiniBand HDR fabrics. The study specifically focuses on the industry-standard motorbike tutorial case, detailing how the fabric impacts application run time and scalability. The results indicate that a single 100Gbps rail of Cornelis OPX performs up to 9% faster than the NVIDIA InfiniBand HDR at 200Gbps in a 16-node cluster. Additionally, the report discusses price-performance metrics, showing that a Cornelis OPX connected cluster achieves an average of 1.57 times higher job throughput per fabric cost compared to the NVIDIA solution. The findings suggest that users can achieve optimal OpenFOAM performance with a lower budget or enhance computational capacity within the same budget.