Much of the current HPC community is behaving, excuse the gross stereotyping, a little like a dinosaur, wondering what that large red fireball in the sky is. Why? Look at the steady growth of public cloud adoption. Now consider the inevitable improvement in technical capabilities and the volume pricing that large cloud vendors can offer, should they choose. It has already eaten a goodly portion of the enterprise market, so why do we persist in believing that cloud won’t eventually eat the HPC market from the bottom up? It’s a re-run of ‘Attack of the killer micros’ from the early 90’s.
As an industry we are approaching an inflection point, where the less demanding HPC applications will move more rapidly to the cloud. As these users migrate, the money that they bring and which is used to subsidise the purchase of more capable systems for the minority, will start to disappear and the business case for on-premises HPC will become harder to make. The reduction in total addressable market will mean another round of consolidation in the HPC market and a consequent reduction in vendor choice.
“Your free lunch will soon be over. What can you do about it? What are you doing about it?” —Herb Sutter
This soundbite was given in the context of the advent of multi-core and the need for more concurrency in software. We think the same is now true if your application relies on very low latency communication between nodes and that this will become an increasingly ‘taxed’ feature, since today it isn’t a mandatory requirement for cloud. Arguably it may never be.
The ‘free lunch’ is over, so start packing your own. What do we mean by ‘pack your own’ you ask? Essentially we are looking at another round of upheaval in how we best architect codes to run in a scalable fashion on future architectures. If you rely on bleeding edge communication latencies for your MPI stack then you are backing an evolutionary niche. It will have all the same pressures that over specialised or restricted habitat animals are finding in the era of the sixth great die-off.
Don’t be a dinosaur!
