HPE to build Discovery & Lux computing systems for Oak Ridge lab

Hewlett Packard Enterprise (HPE) will build two new computing systems-Discovery and Lux-for the US Department of Energy's Oak Ridge National Labouratory, focusing on advancements in scientific research and artificial intelligence.

The main system, Discovery, will be a second-generation exascale supercomputer based on the HPE Cray Supercomputing GX5000. This successor to Oak Ridge's current Frontier supercomputer is expected to increase productivity by up to ten times for select applications, supporting a range of scientific disciplines including precision medicine, cancer research, nuclear energy, and aerospace.

Discovery's architecture will include the new HPE Cray Supercomputing Storage Systems K3000, which integrates Distributed Asynchronous Object Storage (DAOS) open source software. The system will also enable testbed capabilities for quantum computing, and aims to provide a unified platform that brings together AI and high-performance computing (HPC) to accelerate research outcomes and improve security.

In parallel, HPE will construct Lux, an AI-focused cluster based on the ProLiant Compute XD685 platform. Lux is designed as a liquid-cooled, GPU-powered AI system that will offer multi-tenant, cloud-like access for researchers across the country, supporting AI training and inference workloads. The system will feature AMD Instinct MI355X GPUs, AMD EPYC CPUs, and AMD Pensando networking technology, and seeks to improve access for researchers pursuing machine learning and artificial intelligence initiatives.

System capabilities

According to HPE, the Cray Supercomputing GX5000 powering Discovery is tailored to optimise density and operational efficiency. Discovery will require less data centre space per rack compared to previous installations, using 25 percent less floor area per 900 mm rack versus the prior 1,200 mm EX4000 cabinets.

The high-performance interconnect is provided by HPE Slingshot, facilitating high bandwidth and low latency communication essential for HPC, machine learning, and advanced analytics workloads. In terms of storage capability, the K3000 storage system is expected to deliver up to 75 million input/output operations per second per storage rack-a marked increase over the 18 million IOPS per rack delivered by the existing systems supporting Frontier.

The compute infrastructure for Discovery will incorporate next-generation AMD EPYC processors, codenamed "Venice," and AMD Instinct MI430X GPUs. The entire platform will be fully liquid-cooled, reflecting five decades of HPE's supercomputing and liquid-cooling technology, as first implemented in the Cray-1 in 1975.

HPE's unified management solutions will oversee the infrastructure and applications across the systems, supporting Oak Ridge National Labouratory's objectives in scientific innovation and security.

Stakeholder perspectives

"When we built Frontier for Oak Ridge National Labouratory and ushered in exascale, we achieved the pinnacle in supercomputing history and a triumph for the U.S. We are proud to build on that leadership innovation and strong public-private partnership with the U.S. Department of Energy, ORNL and AMD, to build Discovery and Lux, accelerating the next era of scientific discovery and AI innovation," said Antonio Neri, President and CEO at HPE.

Bronson Messer, Director of Science for the Oak Ridge Leadership Computing Facility, commented on the impact of the new systems, stating, "We are excited for Discovery and Lux to expand the science that researchers are able to do at Oak Ridge. Discovery will set the stage for a new level of converged HPC, AI and quantum computing capabilities, providing additional insight in connection with other systems, while Lux greatly expands researcher access to dedicated AI resources. As a result, we expect both systems will contribute to a paradigm shift in our productivity, reaching unparalleled gains in various, critical areas of scientific research and leadership."

Dr. Lisa Su, Chair and CEO at AMD, noted the company's multi-year role in such systems, saying, "For more than a decade, AMD and HPE have partnered to push the limits of high-performance computing, delivering solutions that enable discoveries and change the world. Together with Oak Ridge National Labouratory, we are advancing the next generation of AI systems with Discovery and Lux-empowering researchers to accelerate innovation and strengthen America's leadership in science and technology."

Scientific and operational advancements

The Discovery supercomputer is intended to support both physics-based simulations and data-driven AI models, linking the lab's HPC and AI research workflows. The system's DAOS-based storage is currently ranked at the top of the global IO500 storage benchmark, highlighting sector-leading performance for data-intensive workloads.

Discovery's liquid-cooled infrastructure is designed to enhance energy efficiency and offer cost-effective supercomputing for sustained operation, complementing the lab's existing Lustre-based storage with new all-flash K3000 racks.

Lux, with its cloud-like, multi-tenant AI platform, is expected to broaden national researcher access to high-performance AI tools for both training and inference, contributing to ongoing AI and machine learning initiatives at Oak Ridge and across other institutions.