Avoiding Potential Risk of Stagnation in the Secondary Storage Market

By Guest Blogger Peter Faulhaber, former president and CEO, FUJIFILM Recording Media U.S.A., Inc.

The Hyperscale Data Center (HSDC) secondary storage market is quickly emerging, requiring advanced solutions for petascale and exascale storage systems, not currently available. According to HORISON Information Strategies, HSDCs currently use around 3% of the world’s electrical energy. Due to the massive energy footprint of HSDCs, climate protection measures have become increasingly important in recent years, with cloud computing offering the greatest advantages for sustainable operation by reducing the energy and carbon footprint over the entire data life cycle.

The slowing rate of HDD and tape technology development roadmaps in recent years, along with HDD and tape storage supplier consolidations are particularly concerning trends to HSDCs. Neither HDD nor tape technology is currently positioned by itself to effectively meet the enormous HSDC storage requirements that future performance and capacity demands. High technical asset specificity requires significant R&D investment, yet have limited ROI potential outside of hyperscalers.

HSDCs manage over 60% of the world’s data today with a CAGR of 35 – 40%, with a growing need for cost-effective secondary storage that still meets certain performance thresholds.

The vendors and manufacturers are dis-incentivized to invest in novel technology; the risk reward is not high enough, while HSDCs are leveraging their buying and bargaining power. Manufacturers need to invest hundreds of millions to bring innovative solutions to market in a long development cycle, without a commitment from the HSDC market.

As a result, the secondary storage market is left with incremental investments in existing technologies and moves slowly.

The conditions are set for a widening gap between customer demands and product solutions in the secondary storage market.

The current “vendor-driven” strategy will not keep pace with HSDC requirements for secondary storage as such offerings fall far behind HSDC curves. Photonics, DNA, glass, and holographic experiments are attempting to address the market, and have been in labs for decades, but most have drawbacks, and none are on the near-term horizon for customer deployment. These initiatives show that a change is needed to get ahead of the demand curve.

However, the opportunity also exists to mitigate this risk by bringing the interested parties together  to share the risk reward paradigm. HSDCs need a quantum leap, which only comes with significant investment, best shared by the interested parties.

The Semiconductor Research Corporation (SRC) addressed the concept  of vertical market failure in September 2021 in its published article “New Trajectories for Memory and Storage,” stating, “The prospect of vertical market failure can be mitigated by private sector market participants through risk-share agreements between customers and suppliers, as well as increased vertical integration.”

Without change, current technologies will fall far behind HSDC demand curves, and the current vendor-driven trajectory increases the likelihood of un-met demand and stagnation of growth for all involved.