New Report by John Monroe of Furthur Market Research a Wake-up Call for the Storage Industry

John Monroe, a long-time storage industry expert and Gartner analyst turned independent consultant, recently published a new report entitled “The Escalating Challenge of Preserving Enterprise Data”. The report, co-sponsored by Fujifilm and Twist Bioscience, looks at the supply and demand for SSD, HDD and tape technologies from 2022 to 2030. The findings and conclusions in John’s report are surprising to say the least and should serve as a wake-up call for executives in both the end user and vendor communities. Below are some summaries and excerpts taken from the report and a link is provided to view/download the full report.

In a Nutshell
John estimates that maximum production capacities for enterprise SSDs, HDDs and tape will likely fall short of storage demand if annual data growth exceeds 25% between 2022 and 2030. This scenario is not unlikely, given the historic CAGR of 30.5% between 2010 and 2021. But what is more alarming is that growth rates could easily hit 35%, 40% or even 45% in the years ahead as there appears to be no imminent reduction in the escalating zettabytes that are being generated by the vast networks of data centers, consumers and sensors. In addition, the retention periods for data are extending beyond predictable periods of 5, 10 , 15, or 25 years to “indefinite” in many cases.  Taking a somewhat conservative forecast of 35%, the industry could be facing a shortage of storage capacity on the order of 7.9 ZB by 2030 with an installed base that has ballooned to 62.9 ZB, or 691X compared to 91 exabytes in 2010. Finally, the cost of petabytes delivered in 2030 could exceed a staggering $121 billion, up from $34.6 billion in 2022. Storage industry financial and business executives should immediately and generously fund the enhancement of old enterprise technologies and the creation of new enterprise technologies that can be deployed more cost-effectively at massive scale with minimal power consumption.

Resurgence in Tape Shipments
More and more tape will be needed to take pressure off required SSD and HDD capacities, as well as other forms of ultra-low cost, massive capacity enterprise storage technologies such as DNA data storage or new breeds of optical media. An estimated 75% of the installed base will be classified as unstructured and cold, ideally suited for tape and DNA. There will be a resurgence in tape shipments for a variety of reasons based on expanding demand on multiple fronts including hyperscale markets, relative data temperature, time-to-data needs, and lower costs of data retention and power consumption, as well as limited SSD and HDD production capabilities. Unlike SSDs and HDDs, tape is largely impervious to malware attacks with “offline air-gap” protection, encryption and write-once, read many (WORM) functionality.

Power Consumption and Long-Term Costs
End user clients are now analyzing storage vendors with greater scrutiny and making purchasing decisions based on the sustainable energy efficiency of a vendor’s products. Because of the rapid expansion of digital data needs, storage as a percentage of data center energy consumption will continue to expand and could account for more than 35% of total data center power requirements in 2030, up from  ~18% in 2020. End users should carefully consider initial acquisition costs, the timing of available capacities in the context of ongoing reliability metrics, performance and power requirements, and the total expense of preserving enterprise data over 5, 10, 20, 50 years and indefinite retention periods.

In Conclusion
The data centers of the future will need everything the SSD, HDD and tape industries can manufacture and deliver, as well as requiring new DNA and optical and perhaps other enterprise storage technologies, to cost-effectively and reliably preserve the priceless artifacts of our personal, corporate and cultural history. Inevitably and inescapably, richly varied computing technologies will come and go, but the data we create will remain, and grow to unimaginable immensity.

You can read the full report here: The Escalating Challenge of Preserving Enterprise Data