Accelerate HPC Optimisation Through Workflow Benchmarking

Most obviously, it allows you to understand the performance of an application or workflow on a particular system (including the per application characteristics which hit or create system bottlenecks).

Having a performance baseline for a range of applications allows comparison of performance or capability between different systems (including prospective replacements as part of a procurement activity) or simply help in identifying misconfigured installations.

Having a performance baseline for a range of applications improves the ability to identify unexpected performance regressions after software security patches, firmware, and operating system updates and associated changes in software dependencies (such as libraries and runtimes).

Benchmarking HPC applications allows you to track performance improvements or regressions over time which can help with prioritising RSE efforts in the areas that will provide the most benefit.

Functional testing ensures that the answer is correct (an important initial step when optimisation of an application has been undertaken) while performance benchmarking measures the time taken (to produce a correct answer – which you should still check btw) for different configurations under test (eg number of cores, parallelisation strategies incl. MPI threads etc).

Benchmarking in HPC really looks at system level performance, with the emphasis on producing a balanced system (compute, memory, fabric and storage) which runs well under a variety of workloads and conditions.

For a variety of reasons High Performance Linpack (HPL) is one of the most common benchmarks but it is perhaps the one least suited to testing a system to simulate normal operation. The best benchmarks are usually the applications which consume the most cycles on your current system, since improvements in runtime will likely increase productivity and allow improvements to the scientific fidelity of results.