HPC cloud belches from top UK universities

At last, one can rent tens of thousands of cores-worth of HPC teraFLOPS on demand from an HPC cloud.

It's called CORE and is based on high-performance computing (HPC) labs at Cambridge University and Imperial College and the STFC DiRAC national HPC service. Users can get access to over 300 teraFLOPS of sustained, double-precision HPC performance, with more than 22,000 Intel cores and a 3PB highly parallel file system.

Cambridge University has run its HPC facilities on a pay-for-use basis for six years and the business model is mature.

CORE includes several systems; the largest and most powerful X86 HPC cluster in the UK, with over 9,600 cores [number 93 in the supercomputer Top 500], the largest single shared memory space, with 2,000 cores and over 16TB of RAM, a 1PB Xyratex ClusterStor Lustre parallel access storage system, and one of the UK's largest NVIDIA GPU clusters.

Customers get access to HPC hardware, a data analytics infrastructure and support and consultancy. Industry can now use CORE for HPC and big data analytics applications. CORE is a component of a UK government Department of Business, Innovation and Skills (BIS)-led e-Infrastructure expansion exercise to get HPC and big data facilities made available for industry and academia so as to enhance overall UK competetivness.

The system has been used by customers already, such as Audio Analytics for sound research and Xyratex for Lustre appliance development. Other customers include Rolls-Royce and the Caterham F1 team; Lotus as was. They all obviously found it was much cheaper to hire time on CORE than set up their own HPC system. They also needed HPC facilities because, without them, it would taken longer to develop products and bring them to market.

Audio Analytics makes sound recognition software. Its founder Chris Mitchell said; "Audio Analytics has harnessed the power of CORE to develop sound packs that can be used by the physical security industry to identify sounds such as breaking glass or gun shots. [CORE] provided us with a step change in compute capability increasing the scope of what we can achieve and reducing time to market."

Xyratex supplies storage arrays to CORE and the Cambridge HPC centre, and used CORE services to develop its ClusterStor Lustre appliance. It was able to profile HPC workload and user application IO behaviour and test ClusterStor performance with IO-intensive workloads.

Both Imperial College and Cambridge University need great HPC facilities to power their own research and CORE income will help them get that.

CORE co-director Dr Paul Calleja of Cambridge University said; "CORE lowers the barriers of uptake for users and organisations new to HPC, removing the necessity for specialist HPC staff and costly in-house IT infrastructure."

HPC Cloud services are provided by CORE as vanilla HPC cycles with system support through a single point of entry. Code can be uploaded through the Internet. Initial starting data sets, if small, can be uploaded in the same way or couriered over, which is what the Caterham F1 team did, using a USB disk and motor bike rider.

Application-generated data, which can range up to tens of terabytes in size, can be analysed in the CORE HPC machine rooms and remotely visualised, as transferring that amount of data to customers by network means would be very very slow, and a motor cycle courier would still involve delay.

Solution packages are available for engineering, life sciences, materials modelling, and digital media remote render farms. CORE consultancy packages exist to help customers set up their own in-house HPC facilities if they wish to do so. There are free starter packs including up to 10,000 core hours and one day of support. Find out more about CORE here. ®