It’s easy to be distracted by news from the US, China, and now the EU on the state of various exascale projects, but behind the vinyl-wrapped cabinets and well-groomed sales execs are an army of Excel-wielding PMO and accountancy staff trying desperately to keep projects on-time and on-budget. Applying the same level of scrutiny once the system is installed is sadly somewhat rare, however.
One argument we often hear from staff at research-focused organisations is that they shouldn’t need to justify their expenditure on HPC in great detail – after all, supporting fundamental research is part of the core mission of the institution, and no-one is doubting the role computational work plays in that effort. Ultimately though, universities and public-sector research organisations are increasingly being driven towards more business-like operational models, and so HPC may need to follow suit. Since cloud HPC is only just barely starting to gain acceptance as a viable option, providers of on-premise HPC could do with a way of insulating themselves from the scrutiny of accountants who don’t concern themselves with the details of why a bare-metal solution is still the preferred choice of the user community.
It has always been the case that those responsible for paying for HPC have a vested interest in exploring alternative sources of revenue, particularly by squeezing out wasted cycles. It isn’t that they don’t trust their research staff to deliver value – investment in HPC has been shown time and again to deliver a strong return, with numbers up to 500x ROI commonly quoted. Seeking alternatives to the normal user-base isn’t about increasing income per se, but rather about diversifying where that income originates from as a hedge against sudden changes in funding poilcies or unexpected drops in utilisation.
Let’s take university HPC funding as an example. Deciding how much to spend on a HPC system is generally a simple function of how much capital is actually available, and how much research income the university expects the system to generate. If the university were to invite paying commercial customers onto their system, they could augment their investment and create opportunities for collaboration at the same time – surely a win-win?
Sadly, it isn’t all that easy. Creating a service which meets the higher standards of industry in terms of data protection and reliability can quickly make the proposition unaffordable, particularly now that the customers can obtain pay-as-you-go computing resources from cloud vendors, which readily offer the sort of SLAs a company would expect when paying for a service. It is for this reason that Red Oak generally advises it’s university customers to focus their HPC outreach efforts on delivering value to partners through the expertise of their staff, rather than their unused compute capability. So, without bringing commercial users on board, how could a research HPC system owner diversify their income sources?
Cryptocurrencies have created an odd effect in the computing world whereby idle cycles have suddenly become quite valuable – it is now worthwhile for hackers and unscrupulous websites to mine cryptocurrency in your browser, sometimes justified as an alternative to presenting ads or harvesting your data.
Based on recent trends in mining difficulty and price, we can estimate that mining a cache-bound altcoin will typically deliver in excess of a 2x return on the cost of electricity, but will not come close to meeting the TCO for a HPC system. Relative to just running research then, we can determine that it might be worthwhile filling up unused nodes on your cluster with low-priority mining workloads, which can easily be pre-empted by “real” compute jobs.
There is an obvious, and dangerous, corollary to the above; while we are deriving some sort of tangible benefit from this mining activity, why not just bump up the priority a bit to exclude those users whose research doesn’t bring in any grant income? This is the sort of thing which might seem like an easy win to someone with too much focus on the balance sheet, but clearly neglects the benefits which speculative research activities can offer down the road. Companies whose HPC activity is defined by a rigid set of workflows might be able to justify the more mercenary approach, but will ultimately suffer the same long-term negative impact on innovation if they focus too heavily on squeezing value out of their hardware by brute force.
Once a HPC system owner has generated a stack of digital currency, they are faced with a new problem – sell or hold? This is where things become even more interesting; selling the currency immediately would cover the extra electricity costs which are being generated (assuming someone is occasionally tracking prices, and switching off the miners when they become unprofitable). As the safe and sensible (read: boring and unimaginative) option, this will appeal to some managers who are already nervous about the whole cryptocurrency business, but those who are more bullish could instead focus on convincing their senior stakeholders that this kind of high risk, high reward “investment” has greater long-term gains to exploit. They should be careful though; if too much staff time is spend tracking prices and making investment decisions, the miners might quickly find their profits wiped out by decreased productivity!
Besides the potential to act as an enormous distraction, institutions should also consider that getting involved in cryptocurrency might carry some reputational risk. The widely adopted variants which are suitable for CPU mining (and hence more appropriate for mining on HPC) tend to emphasise privacy, naturally leading to a large uptake within criminal circles. Furthermore, while turning electricity directly into money might have a good business case, it doesn’t exactly fit well within any legitimate “green” agenda which might be in effect.
One final message which anyone considering the addition of mining to their HPC repertoire ought to heed – remember who owns the coins! Earlier days of cryptocurrency saw mining occasionally being done on HPC systems, but back then it was rogue users looking to pocket the income rather than administrators with a plan and, more importantly, approval.
Whether this extra monetisation scheme is a good idea or not depends more on the risk appetite of the institution and the availability of good ROI calculations than on any technical quirk of the setup. Every HPC system manager ought to be evaluating their TCO against the benefits their service provides; the potential for monetising wasted cycles need only be one more term in the equation.
Chris Downing – Senior Consultant
