Archival data may not require high performance computing, but it does need to stay accessible for productivity, legal, business value, analytics, and compliance. IT must secure that data against ransomware and other types of cyberattacks. And IT needs to do all this at reasonable costs, even given extraordinary projected data growth.
Active archiving technology is constantly evolving as marketplace demand increases. We recently released our 2022 Report: The Active Archiving Ecosystem: Building a Flexible Archival Repository Your Way, highlighting the increased demand for new data management strategies and the benefits and innovations of active archive solutions.
Top innovations within active archiving include Artificial Intelligence and Machine Learning (AI/ML), sustainability, analytics, and compliance.
The LTO Technology Provider Companies (IBM, HPE, and Quantum) issued a press release earlier this week announcing record capacity shipments for LTO in 2021 of 148 Exabytes (148,000 Petabytes) compressed (up from 105 EB compressed in 2020, +40%). More and more of the world’s data is being stored on LTO data tape. That’s good news for the IT industry! Is it not? After all, end users and service providers need:
A strategic way to store and protect massive amounts of increasingly valuable data, especially data that’s gone cool or cold
A cost-effective and reliable long term storage solution
An air gap defense against cybercrime
An eco-friendly form of storage!
Industry Pundits React Some industry pundits, biased toward the HDD industry, took the opportunity to downplay the news. They said the data is inaccurate or insignificant compared to the capacity shipments for HDD last year. Really? Does tape technology threaten the market for HDD? Is it still about tape vs. disk in their minds? Have trains, trucks, and ships put air freight out of business? Or does a more strategic thought process say: “These technologies complement each other. We need both to meet the needs of end-users, storage service providers, and society itself…”
Analysts Predict Huge Zettabyte Demand Indeed, if the big industry analysts firms are right, we will need to be storing more than 11.0 zettabytes of data in 2025. Just one zettabyte would require 55 million 18.0 TB HDDs or 55 million LTO-9 tape cartridges. Should we store all of that data on HDD, whether it is hot, warm, cool, or cold? Of course, we can’t just delete excess data. Now that we can analyze the data and derive a competitive advantage from it, the value of data has increased and we need to store more and more data for longer periods of time. As a result, the projections for the amount of persistent data to be stored are growing exponentially. We will need huge amounts of flash, HDD, tape, and even future storage solutions like DNA to address the data storage challenge.
A Strategic Approach to Data Storage The key to success will be a strategic approach that leverages intelligent data management software to automate data movement to the right tiers of storage at the right time, the right cost, and the right energy profile. Employing a strategic approach to data storage in an effort to reduce costs and energy consumption all while maintaining service level agreements seems to make sense. Take a good look at an active archive solution, for example. Yet again, there are those industry pundits who say, the amount of energy saved by moving static, inactive, and infrequently accessed data to a tape tier is not significant in the big picture of the IT industry. The real problem they say is the amount of energy consumed by a single Google search. But isn’t that like saying; “Don’t bother turning the lights out before leaving the office for the night. It’s just a drop in the ocean of energy consumption,” or “Why bother turning off the engine of your car when filling up on gas? It’s just a few minutes of idle time and won’t really impact CO2 emissions at all.” Right?
Change of Attitude Needed But this is the wrong attitude and exactly what has to change to make a difference. Collectively, if we all switch off a light and all turn the car’s engine off, we will make a difference. We might even get motivated for more change! How about installing LED light bulbs or investing in an electric vehicle? Or maybe make the commitment and take the leadership on a renewable energy installation. Attitudes have to change, believing we can make a difference collectively. If data is inactive, why keep it on energy-intensive, constantly spinning disk? Are we all doing whatever it takes to make a difference?
New Flagship UN Report Is a Wake-up Call If we believe the latest studies on climate change coming out of the United Nations, we need to start quickly taking any action we can. A new UN report on climate change from earlier this month indicated that harmful carbon emissions in the last decade have never been higher in earth’s history. It’s proof that the world is on a “fast track” to climate disaster. UN Secretary General Antonio Guterres has warned that it’s ‘now or never’ to limit global warming to 1.5 degrees C. Climate change is the result of more than a century of unsustainable energy and land use, lifestyles, and patterns of consumption and production. Guterres adds, “This is not fiction or exaggeration. It is what science tells us will result from our current energy policies. We are on a pathway to global warming of more than double the 1.5-degrees C limit” that was agreed in Paris in 2015. To limit global warming to around 1.5 C (2.7 F), the IPCC report insists that global greenhouse gas emissions will have to peak “before 2025 at the latest, and be reduced by 43% by 2030.”
Reducing Energy Consumption and CO2 Emissions with Tape To help increase awareness and understanding of energy consumption in data storage, a number of whitepapers have been published highlighting alternative options for storage including LTO data tape. A recent IDC whitepaper studied migrating cold data from HDDs to LTO tape. The opportunity to positively impact the environment by shifting to tape is staggering. This strategic approach can reduce storage-related CO2 emissions by, coincidently, 43.7% by 2030. This would avoid 664 M metric tons of CO2 cumulatively. That’s the equivalent amount of CO2 produced by 144 million passenger cars driven in the course of a year!
Other research shows that tape consumes 87% less energy than equivalent amounts of HDD storage. When CO2 emissions are analyzed over the entire product lifecycle (from raw materials to production to distribution, usage, and disposal) of HDD and tape, studies show a 95% reduction in CO2 in favor of tape compared to HDD. The same study shows Total Cost of Ownership for long-term data storage can be reduced by more than 70% by using tape instead of HDD. At the same time, tape can provide an effective defense against cybercrime via a physical air gap. All of this is possible by taking a strategic storage approach, where cool or cold data that has aged and is infrequently accessed gets moved from expensive primary storage to economical and environmentally friendly tape systems, online or offline.
Data Center World Attendees Get It In my last blog on my visit and presentation at Data Center World in Austin last month, I mentioned that I was encouraged by the DCW attendees that I met and listened to in my session and other sessions. They are genuinely concerned about the environment and worried about what kind of planet we will be leaving behind for our kids and grandchildren. They recognize the opportunity to improve sustainability in data center operations and are committed to it. But since then it has occurred to me that maybe sustainability is more of a focus for facility teams. Perhaps the top-down pressure from the C-suite has yet to be widely applied to the data storage management teams. However, in the quest to achieve the needed sustainability goals, no stone can remain unturned.
Observing Earth Day for Future Generations With Earth Day being observed today, let’s commit to strategically taking action in response to global warming and climate change. Let’s start changing attitudes from “It won’t make a difference” to “Collectively, we can make a difference.” Let’s look at the bright side of increasing LTO capacity shipments instead of the dark, self-serving side. Let’s think about the planet that’s home for us and the future generations of our families to come.
I think it’s safe to say people like surveys, probably not everyone, but most people do. Why? Experts in the field suggest that people are willing to take surveys because respondents feel their opinions are valued and that their answers will be used and may even result in a benefit to society. They feel their participation will impact something they care about, and they want to share their opinion with those who will listen and act on the information.
Surveying the C-Suite on Sustainability So it’s not surprising that Fujifilm got a great response rate to a recently launched survey entitled Awareness Survey on Environmental Issues in the Digital Domain. As many as 1,200 C-suite professionals responded including CEOs, CFOs, CSOs, CTOs, and CIOs from companies of 100 or more employees in the United States, Germany, Japan, and China.
The survey revealed that there is a growing awareness around broader environmental issues among corporate leaders, and that’s great news as the negative impacts of global warming and climate change keep piling up, flood after flood, wildfire after wildfire, and storm after storm.
When it comes to IT infrastructure specifically, the majority of U.S. respondents believe sustainability improvements in IT services and equipment can positively impact climate change, but 40% indicated that they did not know or were unsure if data storage can have a negative environmental impact and increase the cost of doing business.
Increasing Data Storage Requirements Data storage can certainly be energy-intensive. This is a problem that is only getting worse as the value of data rises with the ability to analyze and derive competitive advantage from it. As a result, demand for long-term data retention is increasing. In fact, data to be stored is expected to grow from just 2.0 zettabytes in 2016 to 4.1 ZB in 2020 and is expected to reach 11.1 ZB in 2025 according to a recent whitepaper from IDC. Just one ZB is a vast amount of data equal to one million petabytes that would need 55 million 18 TB hard disk drives (HDDs) or 55 million 18 TB LTO-9 tapes to store. The environmental impact of the energy required to support this volume of storage is greatly underestimated, as are the associated carbon emissions. When asked in the survey what barriers exist for those who have not considered more eco-friendly data storage options, 31% in the U.S. cited a lack of awareness or understanding of the issue.
Hot vs. Cold Data There was also a lack of awareness pertaining to frequently accessed “hot” data and less frequently accessed “cold” data, with 36% of respondents saying they either don’t or are unsure if they differentiate between the two. And 35% don’t realize that differentiating between hot and cold data can impact sustainability, affordability, and security. An interesting fact about data is that it quickly goes cold and access frequency drops off significantly after just 30, 60, or even 90 days. In fact, industry analysts estimate that 60% to 80% of all data stored is cold and qualifies as “archival”. Yet through inertia, that data often remains on energy intensive, constantly spinning and heat-producing tiers of storage like hard disk drives.
Reducing Energy Consumption and CO2 Emissions with Tape To help increase awareness and understanding of this issue, a number of whitepapers have been published highlighting alternative options for storage including LTO data tape. A recent IDC whitepaper shows how migrating cold data from HDDs to LTO tape can reduce data centers’ CO2 emissions by 43.7% by 2030, avoiding 664 M metric tons of CO2 cumulatively. Other research shows that tape consumes 87% less energy than equivalent amounts of HDD storage. When CO2 emissions are analyzed over the entire product lifecycle (from raw materials to production to distribution, usage, and disposal) of HDD and tape, studies show a 95% reduction in CO2 in favor of tape compared to HDD. The same study shows Total Cost of Ownership for long-term data storage can be reduced by more than 70% using tape instead of HDD. All of this is possible by taking a storage optimization approach, where data that has aged and is infrequently accessed, otherwise known as cold data, gets moved from expensive primary storage like solid-state flash drives and HDDs to economical and environmentally friendly tape systems.
As far as security is concerned, tape is also playing a role in cybercrime prevention with air gap capabilities, WORM, and encryption. Intelligent data management software, typical in today’s active archive environments, can automatically move data from expensive, energy-intensive tiers of storage to more economical and energy-efficient tiers based on user-defined policies. By moving inactive data out of primary storage, the ransomware attack surface can also be reduced.
Renewable Energy Plus Conservation Another interesting point from the survey reveals that 51% of participants said that their companies are using renewable energy to reduce carbon emissions, while 22% said they are doing so via climate protection projects and 13% through carbon offsets. Renewable energy is a key factor in reducing CO2 emissions and Fujifilm is a fan (see photo at right of our LTO plant in Bedford, MA). But alone renewables likely can’t come online fast enough or cheaply enough to keep up with data growth rates of between 30% – 60% annually in major data centers today. That’s why conservation has to be part of the equation. The very first metric to be analyzed in data center energy efficiency is simply the amount of energy that’s being consumed.
Alternative Data Storage Options Finally, 81% of respondents noted that they would consider an alternative data storage option that is more sustainable and affordable. That option exists in the form of today’s modern and highly advanced data tape systems that offer the lowest energy consumption and cost profile. Add to that its best-in-class reliability rating of any storage media and longest archival life. So for the benefit of society, let’s act on the information that the survey reveals. It’s really just a question of getting the right data, in the right place, at the right time.
It seems like 2020 and 2021 have blended to combine into one long, tough time for all of us. Let’s hope 2022 emerges on the brighter side! In the meantime, here are 5 big predictions we see coming up in this New Year and beyond:
1. Increasing Focus on IT Energy Consumption
Severe weather was once again a hallmark of 2021, from the Texas deep freeze right up to the bitter end of 2021. As unusual tornadoes and wildfires reminded us of the negative impact of global warming and climate change.
According to a report from the United Nations released in August of 2021, irreversible damage has already been done to the environment as a result of greenhouse gas emissions. The world showed renewed interest in the COP 26 conference in Glasgow where countries from around the globe gathered to pledge their commitments to combat climate change.
Wall Street got in on the act too and will increasingly demand that companies disclose their sustainability initiatives and results. Accordingly, more and more companies will be appointing Chief Sustainability Officers who will put pressure on their organization’s energy usage including energy-intensive IT operations. The use of renewables, but also energy conservation measures will be mandated.
Curbing CO2 emissions is quickly becoming a C-suite imperative and storage will not escape the scrutiny. Research shows that 81% of CIOs would consider alternative data storage options that are more cost-effective and sustainable. This will set the stage for new tape system deployments that not only can reduce TCO by more than 70%, but can reduce CO2 emissions by 95% compared to traditional HDD storage.
2. Return to Hybrid Cloud Strategies
Prior to COVID 19, the term “cloud repatriation” appeared often in the press as it turned out that cloud was not a panacea for everything. But COVID 19 understandably created short-term storage strategies often resulting in a flight to the cloud.
However, long-term thinking will favor hybrid cloud strategies where the best of public cloud plus on-prem private cloud provides maximum flexibility and value. This will especially apply to data accessibility, regulatory requirements, data governance, and cybercrime risks including ransomware.
Today’s modern automated tape solutions will provide the advantages of cost, scalability, reliability, and data protection to support the hybrid cloud model.
3. Storage Optimization Will Be Key to Data Growth Management
With the continuing digital transformation comes the zettabyte age of storage where data to be stored globally will approach 6.0 zettabytes (ZB) in 2022, according to a leading IT industry analyst. Just one ZB would require 55 million 18.0 TB HDDs or 55 million 18.0 TB LTO-9 cartridges!
Storage optimization, that is to say, getting the right data, in the right place, at the right time, and at the right cost will be critical to maintaining competitive advantage.
Intelligent data management will be required, leveraging multiple tiers of storage, active archives, and innovative S3-compatible archive solutions for object storage. Nowhere will this be more apparent than in digital preservation and high-performance computing environments with a simple need to offload expensive object storage to cost-effective tape systems using an S3-compatible API.
4. Continuing Rise of Ransomware
It has been said that ransomware is only in “its infancy” and it’s been said many more times, an attack is not a matter of “if” but “when.” The FBI and CISA have weighed in with this advice:
“Backup your data, system images, and configurations, test your backups, and keep backups offline.”
As ransomware hackers mature in sophistication (and profits), online backups are increasingly being targeted to hamper recovery efforts, including cloud-based backups connected to a network. As a result, the value of affordable, removable, and highly-portable tape will only increase, providing true air gap protection (meaning offline, offsite backups in a secure location).
5. Video Surveillance Content Management
As we predicted last year, data tape has increasingly become a strategic option in managing the ballooning volume of video content associated with video surveillance applications.
Due to security reasons, regulatory compliance, or for future analytics, retention volumes and periods will only increase making legacy HDD solutions cost-prohibitive and unsustainable in terms of energy consumption. Look for increasing adoption of cost-effective tier 2 tape in video retention workflows in 2022.
Successfully emerging from the combined years of 2020 and 2021 will require getting back to strategic, long-term planning. Given the relentless growth of data, environmental concerns, and limited resources and budgets, today’s highly advanced tape storage will play an increasingly vital role in 2022 and beyond.
The 2021 Active Archive Alliance annual market report has just been released, entitled “Saved by the Data. Active Archive Leads the Way in a Mid-Pandemic World”.
Certainly, the COVID pandemic was a shock to many companies and put tremendous strain on operations, revenue, and profit. But those companies who had already implemented a sensible active archive strategy were at a competitive advantage thanks to their ability to intelligently manage access to their data.
I think active archiving, the practice of keeping data online all the time and easily accessible to users, is a hot concept in storage right now because it is really about optimization – getting the right data in the right place, at the right time, and at the right cost.
We know that IT budgets are not keeping up with the relentless growth of data. We also know that 60% to 80% of data quickly becomes archival. Typically after 30, 60, or 90 days, files become static and the frequency of access drops off. So why keep that kind of data on expensive primary storage?
Why not let intelligent data management software that is typical of an active archive solution move that data by user-defined policy from high performance, expensive tiers, to lower performance but more cost-effective tiers like economy disk or tape systems, or even cloud? All while maintaining transparent access for users.
We know that the value of data is increasing, retention periods are getting longer, and users want to maintain ready access to their data without IT staff intervention. But we also need to worry about the bottom line, about efficiency, compliance, sustainability, and cybersecurity! Active archiving provides the right solutions to these worries and that’s why it is such a hot concept in storage today.
But enough said, read the full report here and check out what Alliance members had to say in their related virtual conference.
There is increasing pressure around the world to reduce emissions and lower mankind’s carbon footprint. It is up to the IT sector to do its part, and that means considerably lowering power usage. But that is easier said than done when you consider the statistics.
IDC predicts we will arrive at the mind-boggling figure of 175 zettabytes of data in the digital universe within 4 years. 175 ZB? Consider how long it takes most users to fill a one TB drive. Well, 175 ZB equates to approximately 175 billion TB drives.
The problem is this: how do you reduce IT’s overall power draw in the face of a massive and continual upsurge in data storage? Once 175 ZB of data exists, there is no possibility of containing electrical usage if the vast majority of storage is sitting on hard disk drives (HDDs). The only solution is to cure the industry’s addiction to disk.
Here are the numbers. Data centers alone account for close to 2% of all power consumed in the U.S., about 73 billion kilowatt hours (kWh) in 2020. That is enough to set off the alarm bells. Yet tremendous progress has been made over the past two decades in terms of data center efficiency. When power consumption in data centers soared by 90% between 2000 and 2005 period, the industry acted forcefully. The rate of growth slowed to 24% between 2005 and 2010 and then fell to less than 5% for the entire decade between 2010 and 2020. That’s miraculous when you consider that it was achieved during a period that represented the largest surge in storage growth in history. Smartphones, streaming video, texting, multi-core processors, analytics, the Internet of Things (IoT), cloud storage, big data, and other IT innovations demanded the retention of more and more data.
Big strides were made in Power Usage Effectiveness (PUE – the ratio of data center power consumption divided by the power usage). Data centers have largely done a good job in improving the efficiency of their operations. But the one area lagging badly behind is storage efficiency.
In mid-December 2020, Fujifilm issued a press release to announce that, together with IBM Research, they had successfully achieved a record areal density of 317 Gbpsi (billion bits per square inch) on next-generation magnetic tape coated with next-generation Strontium Ferrite (SrFe) magnetic particles. This areal density achievement would yield an amazing native storage capacity of 580TB on a standard-sized data cartridge. That’s almost 50 times more capacity than what we have now with an LTO-8 tape based on Barium Ferrite (BaFe) at 12TB native.
Shortly after the news came out, I was on a call with a member of our sales team discussing the announcement and he asked me when the 580TB cartridge would be available and if there was any pricing information available yet? He was also curious about transfer speed performance. I had to admit that those details are still TBD, so he asked me “what are the 3 big takeaways” from the release? So let’s dive into what those takeaways are.
Tape has no fundamental technology roadblocks
To understand the magnitude of tape areal density being able to reach 317 Gbpsi, we have to understand just how small that is in comparison to HDD technology. Current HDD areal density is already at or above 1,000 Gbpsi while achieving 16TB to 20TB per drive on as many as nine disk platters. This level of areal density is approaching what is known as the “superparamagnetic limitation,” where the magnetic particle is so small that it starts to flip back and forth between positive and negative charge. Not ideal for long-term data preservation.
So to address this, HDD manufacturers have employed things like helium-filled drives to allow for closer spacing between disk platters that allow for more space for more platters, and therefore more capacity. HDD manufacturers are also increasing capacity with new techniques for recording involving heat (HAMR) or microwaves (MAMR) and other techniques. As a result HDD capacities are expected to reach up to 50TB within the next five years or so. The reason tape can potentially reach dramatically higher capacities has to do with the fact that a tape cartridge contains over 1,000 meters of half-inch-wide tape, and, therefore, has far greater surface area than a stack of even eight or nine 3.5-inch disk platters.
But let’s also look at track density in addition to areal density. Think about the diameter of a single strand of human hair which is typically 100 microns wide. If a single data track on HDD is 50 nanometers wide, you are looking at 2,000 data tracks for HDD on the equivalent width of a single strand of human hair! For tape, with a track width of approximately 1,200 nanometers, you are looking at just 84 data tracks. But this is actually a positive for tape technology because it shows that tape has a lot of headroom in both areal density and track density, and that will lead to higher capacities and help to maintain a low TCO for tape.
But let me make it clear that this is not about HDD vs. tape. We are now in the zettabyte age having shipped just over an impressive one zettabyte (1,000 exabytes) of new storage capacity into the global market in 2019 of all media types. According to IDC, that number will balloon to a staggering 7.5 ZB by 2025. We will need a lot of HDDs and a lot of tape (and flash for that matter) to store 7.5 ZB!
In this Executive Q&A, Rich Gadomski, Fujifilm’s Head of Tape Evangelism, discusses the evolution of tape storage and how recent innovations and advancements in tape technology are putting it at center stage.
Explain your role as Tape Evangelist for FUJIFILM Recording Media, U.S.A.
I believe tape technology has a compelling value proposition and a great story to tell. I want to be focused on telling that story and that is my primary role. Of course, it takes a team and we all do our part, but this is what I’m passionate about and want to focus on.
How has the tape storage industry changed over the years?
Tape technology has come a long way since its introduction in the 1950s where we had reel-to-reel tape that could hold just a few megabytes of data. Today we have LTO tape in the market at 12 TB native and enterprise tape at 20TB. Just looking at LTO which came out 20 years ago, it was LTO Gen 1 in 2000 with 100GB of capacity. We expect LTO-9 soon at 18TB so a 180x increase. And just recently we announced a joint demonstration with IBM showing that we can reach up to 580 TB on a single cartridge with our next generation of magnetic particle called Strontium Ferrite. But it’s not just about capacity increases, tape has also made advancements in transfer rate, now faster than HDD, reliability rating, now three or four magnitudes better than HDD and it’s the greenest form of storage consuming far less energy than constantly spinning HDD and this contributes to its lowest total cost of ownership. But today it’s not about tape vs. HDD, in fact, the two technologies complement each other to help customers optimize their data storage.
Why is tape storage still relevant today? What is driving demand for this technology?
I would say first and foremost the incredible amount of data that’s being created in this digital economy. Add to the fact that the value of data is increasing so we want to store more of it for longer periods of time. Yet, IT budgets are flat or barely increasing, so you need a high capacity, highly reliable storage media that is cost-effective for long-term storage and that’s what tape gives you. So tape plays a role in cold archiving, active archiving and yes, backup as well. In fact, tape has renewed interest from customers in the fight against cybercrime and ransomware since customers can easily backup to tape and remove those backups from the network to an isolated and secure location that hackers can’t get to. That’s what we refer to as the tape air gap. Last but not least is energy consumption. Society is rightfully concerned about climate change. Tape consumes 87% less energy than disk and that leads to 87% less CO2 emissions.
Modern tape storage has long been recognized for its low cost. Several analyst white papers have been published that demonstrate the low cost of storing data on tape. For example, “Quantifying the Economic Benefits of LTO-8 Technology” is a white paper that can be found on the LTO.org website. However, occasionally a storage solution provider publishes a white paper that claims to show that their solution is less expensive than tape storage for a particular use case. A good example is a recent white paper published by a disk-based backup-as-a-service provider who will remain unidentified out of respect for what they do. For the purpose of this blog, let’s call them “BaaS.” So let’s dig into their analysis which makes several assumptions that result in higher costs for tape storage than most users would experience.
Total Cost of Ownership (TCO) Process
The first step in developing a Total Cost of Ownership (TCO) estimate is the determination of the amount of data to be stored. The BaaS whitepaper separates the amount of primary data, which we wish to protect, from backup data, which is the data physically stored on the backup media. They estimate the amount of backup data residing in the tape library to be two to four times the primary data. This is due to their use of the old daily/ weekly/monthly/ full backup methodology for estimating the amount of backup data. The result is that two to four times the amount of primary data ends up being stored on tape, raising the tape hardware and media costs by two to four times.
Organizations across all industries are concerned about global warming and are actively looking for ways to reduce carbon emissions. As a result, many companies have decided they must incorporate carbon reductions into their strategies and have announced green initiatives.
Fortunately for IT organizations, there is a significant opportunity to achieve meaningful carbon emissions reductions while lowering operational and capital expenses by changing the way they store their data. Researchers estimate that data centers consume 1.8% of all electricity in the United States. Within the data centers, data storage is a significant portion of total energy usage and disk systems are the primary driver of storage energy consumption.
It is estimated that up to 60% of stored information is seldom accessed, meaning that the expectation of access diminishes after 30 days. By identifying this “cold data” and moving it to modern tape storage, organizations can dramatically reduce energy consumption and associated carbon emissions while also lowering data center capital and operational expenses.
Usage of Cookies