The all-flash data center is fast becoming a reality

In 2023, every Australian business will be looking for ways to more efficiently manage ever-increasing amounts of data while at the same time seeking to increase efficiency in the face of a tightening economy.

Couple those imperatives with the emerging challenge of reducing carbon emissions (particularly in the eyes of regulators warning about greenwashing) and many businesses must be asking themselves, what actions can we take?

One area which organisations can focus on for economic efficiency and sustainability improvements is in the data centre.

To understand why, we need to review the significant changes which have occurred in data centres in recent years.

In fact, it started back in 2011, when Pure Storage released the first all-flash storage array that was robust and economical enough for mission-critical data centres. Ever since then, Pure has been on a mission to make all-flash data centre a reality and today, this vision is closer than ever to becoming a reality.

But how is flash making a difference to data centres?

Firstly, there has been no question of the advantages of flash over legacy storage medium such as disk or tape. But the main holdback has been cost. In fact, the disk/flash TCO (total cost of ownership) cutover is closer than many people may realise. In this article, we look at the elements that will render hard drives the less cost-effective option in the near future.

It's important to note up front that flash has already been the obvious choice for any performance- or latency-sensitive workload for some time now. Since flash memory's entrance into the enterprise storage arena, it has been knocking down hard disk tiers one by one, from top to bottom, until now, there's really only one tier left where disk drives maintain a market share advantage: nearline, capacity-focused drives.

This is the lowest tier of online data storage, but it also represents a huge portion of bytes being stored, as well as the growing majority (more than 70%) of hard drive capacity shipped each year. That's the tier that flash will penetrate next.

The price-per-bit for NAND flash is declining at a rate much faster than nearline hard drives.

This, combined with the power, space, and cooling savings, higher performance, and better reliability of flash, will soon result in hard drives becoming the less cost-effective option. Let's go into a detailed look at some of these components.

Media Cost

The first-order cost is always going to be the media itself. While today the cost-per-bit of a hard drive is still lower than the densest flash, there are several key trends to pay attention to in 2023.

First, all the major tier 1 flash manufacturers are demonstrating significant density increases this year, with over 200 layers of stacked 3D NAND in some cases. This increase in density will translate into better cost efficiency, as well as further improve power and space savings. Second, if analyst predictions hold, NAND prices overall are expected to decline through most of 2023, continuing a trend that began in late 2022.

Energy Cost

Data centre infrastructure represents over 1% of global energy consumption, and this share is growing. As a result of rising energy costs, energy consumption by information technology has become not just a concern from an environmental standpoint but now also a significant economic one.

When looking at the total cost of ownership, energy consumption will play an even more outsized role in 2023 and beyond, and infrastructure that is more power efficient will have an even greater economic advantage over the competition. As flash density continues to increase, power efficiency will surpass hard drives and thus contribute to a lower effective cost.

In 2022, Pure Storage published our first ESG report, which compared our systems to competitive all-flash alternatives, where we showed that our systems can use up to 84% less energy. Pure's energy efficiency will be a leading factor in enabling Pure Storage products to achieve a lower TCO than disk in the future.

Higher Performance

In two key ways, the flash performance advantage over disk can translate into cost advantages.

First, because disk performance is still so low, even in workloads with modest performance requirements, you will often end up with stranded capacity. While spinning drives may hold 20TB, if the performance of the system taps out at 16TB, you will not derive as effective a cost-per-bit. Flash doesn't have this issue because read performance stays predictable even as drives fill up, unlike hard drives.

Second, any storage environment that focuses on resiliency will have a redundancy strategy which may involve storing multiple copies of data, for example, in addition to keeping spare capacity around in the case of a failure.

This means that the performance of any one device is a variable in the equation used to determine how much redundancy you need to build into your system. Many years ago, hard drives had gotten large enough that after a drive failed, rebuilds would take so long that you needed to adopt dual parity strategies to ensure business resiliency. Today that can be triple parity or more.

With flash devices, faster rebuild times mean less bits need to be dedicated to resiliency structures, which equates to better effective cost efficiency. Some flash vendors, like Pure Storage, offer robust, automatic resiliency tuning that takes all these aspects into consideration without requiring users to determine what RAID level they need to set.

Better Reliability

Pure's data shows that the annualised failure rates (AFR) of our systems are significantly lower than the industry benchmarks for spinning disks, as well as lower than industry SSD averages. If drives are more reliable, this goes into the same resiliency considerations as above.

But beyond that, it also means that more reliable devices need fewer replacements and, therefore, lower costs. While most customers with support contracts don't feel the cost directly, the cost of this unreliability is baked into the cost of those very support contracts. This is one reason why traditional storage vendors' support costs would skyrocket in years 4, 5, and beyond.

This reliability means Pure's customers don't have to undergo disruptive forklift upgrades every 4 to 5 years. 97% of Pure Storage arrays purchased six years ago with an Evergreen subscription are still in service today and are completely up to date from a technology point of view.

Flash Is Just Better

The simple truth is that no one today continues to buy spinning drives because they're better, but only because they're cheaper. In almost every way, flash memory is superior to spinning magnetic storage, and where flash is already better, those advantages are increasing. Where flash is still behind disk is cost, and the road to the disk/flash crossover point is getting shorter and shorter. The dream of an all-flash data centre will soon become accessible for most organisations.