As artificial intelligence (AI) and machine learning (ML) workloads begin to permeate and drive business processes and decision-making at all levels, effective data management is becoming imperative. Data needs to be stored for a long time and it needs to be available to be actively accessed during its lifespan.
Active archive solutions are ideal for managing information for AI and ML frameworks as they can provide customized optimization for storage, security, and performance. In addition, AI tools can be applied in an active archive to automate and streamline data management processes in various ways. For example, AI can cleanse, normalize, and categorize long-term data for AI workloads, automate metadata tagging and indexing for inactive data, and identify and archive sensitive information.
A new eBook from ESG Research, sponsored by the Active Archive Alliance, explores the general state of data archiving and the benefits and challenges shaping modern environments and strategies. It also examines how active archiving integrates into modern data archiving practices and the role it plays in determining the success of AI/ML initiatives.
Key findings include:
On average, organizations have 6.7 PB of data on corporate servers, with 44% of the data on an active archive.
Active archive data growth is in lock step with overall data growth; archive data doubling roughly every 3.5 years.
AI/ML capabilities are a key accelerator for the adoption of active archives.
The top three use cases for active archives are analytics, business efficiency, and cyber-resilience.
Ultimately, AI depends on well-organized data for success, which underscores why effective data management through an active archive is crucial for an AI future. Organizations without intelligent data management processes that feed into business intelligence workloads risk being left behind by competitors who do.
Fred Moore of Horison Information Strategies, a long-time storage industry expert and consultant, recently published a 2024 update of his report entitled “Tape. New Game. New Rules”.
This updated report provides a focus on how modern magnetic data tape is solving for IT challenges including runaway data growth, economic pressure, sustainability issues, cybercrime and the reliability that’s needed for the long term preservation of data. And that data that has also grown dramatically in value as we learn to analyze it and derive competitive advantage from it.
Below are 9 reasons why today’s modern tape systems represent a new game with new rules. Taken together, they make a compelling case for many to revisit the rich value proposition that tape has to offer now and well into the future.
LTO Ecosystem Extends Roadmap
In 2022, the LTO Program Technology Provider Companies (TPCs), HPE, IBM and Quantum Corporation, announced an updated LTO technology roadmap that extends the LTO Ultrium standard through 14 generations. The roadmap calls for tape capacities to double with each new generation, with LTO-14 delivering up to 1,440 TB (1.44 PB compressed) per tape. The new LTO roadmap extension is more relevant than ever and at this point no other storage technologies have revealed a comparable multi-generational roadmap.
LTO-9 Adds Capacity and Features
LTO-9 is the latest LTO generation bringing new functionality to tape including higher capacity, data rate, access time and reliability improvements. LTO-9 increased the native cartridge capacity of LTO-8 by 50% to 18 TB (45 TB compressed) and increased drive throughput (11%) up to 400 MB/sec enabling a single LTO-9 drive to write up to 1.44 TB/hour. A new feature for the LTO family with LTO-9, oRAO (Open Recommended Access Order) reduces initial file access times to first byte of data by as much as 73%.
Record Capacity Achieved with TS1170 Tape Storage System
2023 marked the debut of a new ultra-high-density tape drive with a native storage capacity of 50 TB in a single cartridge and capacities up to 150 TB per cartridge with 3:1 compression. The IBM TS1170 storage system represents the world’s highest cartridge capacity ever announced and enables data intensive secondary storage applications including AI, big data, archiving, cloud computing, and analytics to significantly reduce their total cost of ownership.
Further Improvements Made in Tape Media Longevity
In 2019, Fujifilm and JEITA (Japan Electronics and Information Technology Industries Association) officially confirmed the longevity of Barium Ferrite magnetic signal strength to be stable for at least 50 years based on studies of LTO-7 tapes. Prior to this confirmation, the number of years for LTO tape longevity had been rated up to 30 years.
Tape Leads Reliability Ratings
Since LTO-1 first came to market in 2000 with a native capacity of 100 GB, the capacity of LTO cartridges has increased by 180 times and data rates have increased by 20 times. Over the same period, the specified uncorrectable Bit Error Rate (BER) of LTO cartridges has improved by a factor of 1000, three orders of magnitude improvement. LTO-9 provides an industry leading uncorrectable bit error rate of 1×1020 compared to the highest HDD BER at 1×1017. A BER of 1×1020 corresponds to one unrecoverable read error event for every 12.5 exabytes of data read. Today, both the latest LTO and enterprise tape products are more reliable than any HDD (or SSD).
Tape Reduces CO2, eWaste and TCO
Moore cites key stats from Improving Information Technology Sustainability with Modern Tape Storage, a research paper issued by Brad John’s Consulting that compared an all data on HDD solution to an all-tape solution and to an active archive that moved 60% of the HDD resident (low activity) data to tape. Moving 60% of HDD data to tape for 10 years reduced carbon emissions by 58% and electronic waste was reduced by 53%. Moving 60% of HDD data to tape, results in a 46% TCO savings. Moving all data to tape results in a 78% cost reduction.
Tape Air-Gap Thwarts Cybercrime
The tape air gap, inherent with tape technology, has ignited significant interest in storing data on air-gapped tape. The “tape air gap” means that there is no electronic connection to the data stored on a removeable tape cartridge therefore preventing a malware attack on stored data. HDD and SSD systems remaining online 7x24x365 are always vulnerable to a cybercrime attack.
Data Protection Strategies Evolving with Tape
Using tape to backup HDDs was the original data protection strategy, but having one backup copy is no longer sufficient. The widely accepted and genetically diverse 3-2-1-1 Backup Strategy states that enterprises should have three copies of backup data on two different media types, one copy offsite and one air gap copy. Combining the tape air gap copy with available tape drive encryption and available WORM (Write Once Read Many) tape strengthens any data center cyber resiliency strategy.
Active Archive Leverages Tape
As the amount of secondary storage data soars, new technology tiers are emerging in secondary storage including the Active Archive, Traditional Archive and Deep Archive to address many new use cases. Many data management products now support tape as an object storage target using S3 services. Combining the open tape file system LTFS with tape partitioning, data mover software (HSM, etc.), an HDD array or NAS in front of a tape library creates an active archive.
In Conclusion
At least 80% of the world’s digital data is optimally suited to reside on secondary storage and this amount could reach nearly 7 ZBs by 2025. In response to this, the tape ecosystem has significantly expanded its capabilities in recent years. Tape has also become the leading pure storage solution to defend against cybercrime by seamlessly integrating air gap, encryption and WORM capabilities. Roadmaps signal that the trend of steady tape innovation will continue well into the future. Tape is the greenest storage technology and can significantly reduce carbon emissions and eWaste from data center operations. More large-scale tier 2 data centers are determined to contain their infrastructure costs and improve their sustainability metrics. They will be motivated to rethink existing data storage practices and take advantage of advanced magnetic tape as they approach exabyte scale. Combined with improved access times, faster data rates, a 50-year media life, lowest TCO and the highest device reliability, modern tape has the greatest potential to address the massive capacity demands of the zettabyte era.
Reading Time: 4minutesAs both an active archive and tape evangelist, I’m excited to share how LTO (Linear Tape-Open) tape technology can transform video surveillance storage into a powerful, affordable, and long-term active archive solution. While there is a desperate need for more storage to support the proliferation of video surveillance applications, many in the video surveillance industry view the concept of “archive” as a burdensome process. But when done right with easy-to-use LTO tape systems, it becomes a strategic advantage in the form of an active archive.
The IT industry is often described as “dynamic”. That is to say, constantly changing, and evolving. There is always a “flashy new thing” to grab our attention and distract us. That may be the case when it comes to data storage. With relatively new and flashy things like SSDs, who’s had the time to pay attention to good old tape technology and the its role in ransomware protection?
Actually, lots of big data managers have been paying very close attention to tape technology these days. The largest active archives and archival cloud services on the planet are based on modern generations of tape storage. For everyone else, there is a killer app that is rapidly raising awareness for good old tape and that is ransomware protection.
Leveraging the low data storage cost, removability and portability of today’s highly advanced tape technology turns out to be a great way to get copies of mission-critical data offline and offsite for the inevitable day when cyber criminals take control of an organization’s network and demand a hefty ransom payment.
As National Cybersecurity Awareness Month is upon us, it is worth reviewing the cybersecurity advice and best practices recommended by the government, regulatory agencies and insurance companies. So here are five reasons why data tape backups should be part of every organization’s cyber security and ransomware protection plans:
At the center of an active archive resides an intelligent data management system. This software system plays the central role of automatically placing data where it belongs for cost, performance, and workload priorities. The data management layer uses technologies such as metadata and global namespaces to make data accessible, searchable, and retrievable on whatever storage platform or media it may reside.
Key benefits include:
An essential active archive principle asserts that today’s enterprises need online access to legacy data. Some use cases may need fast access; in other instances, a longer retrieval time is acceptable. The organization determines its access needs and permissions for users and groups.
Cost Savings. Through intelligent data management software, an active archive moves inactive data to low-cost storage. For some organizations, data management software can tier older data to warm storage such as hard drives. Then when, by policy, data has aged sufficiently, files can move to more cost-effective storage such as economy disk, tape, optical or even the cloud.
An active archive can supply a wide range of security features and cyber resiliency capabilities to secure and protect data from cyber threats facing today’s businesses and institutions. Because archival data typically remains unchanged, administrators may use WORM (Write Once, Read Many) or view-only mode features to prevent data from being deleted or overwritten, safeguarding data’s integrity, availability, and confidentiality.
In addition to these benefits, an active archive provides technical leaders the flexibility to adapt to new industry trends and growth areas such as sustainability, artificial intelligence, and edge computing. This flexibility helps enterprises thrive with possibilities for market leadership, increased revenue, and competitive advantages.
To download the 2023 Active Archive Alliance Annual Report, click here.
It’s that time of year again to look ahead at what the New Year might bring in terms of data storage. Looking back at our predictions for 2022 to see how well we did, I think we nailed 4 out 5.
Archival data may not require high performance computing, but it does need to stay accessible for productivity, legal, business value, analytics, and compliance. IT must secure that data against ransomware and other types of cyberattacks. And IT needs to do all this at reasonable costs, even given extraordinary projected data growth.
Active archiving technology is constantly evolving as marketplace demand increases. We recently released our 2022 Report: The Active Archiving Ecosystem: Building a Flexible Archival Repository Your Way, highlighting the increased demand for new data management strategies and the benefits and innovations of active archive solutions.
Top innovations within active archiving include Artificial Intelligence and Machine Learning (AI/ML), sustainability, analytics, and compliance.
After a two year hiatus due to COVID, Fujifilm’s 12th Annual Global IT Executive Summit took place last week in beautiful, warm and sunny San Diego. This year’s Summit theme was “Optimizing storage in the post-Covid, zettabyte age” where organizations have to do more with fewer resources while the value, volumes and retention periods of data continue to increase unabated. It was so good to once again interact face-to-face with members of the storage industry family including around a hundred or so customers, vendors, industry analysts, and storage industry experts during the 3 day event.
About The Summit For those not familiar with the Summit, it is an educational conference featuring presentations from industry experts, analysts, vendors and end users about the latest trends, best practices and future developments in data management and storage. A concluding speaker panel with Q & A and peer-to-peer networking opportunities throughout the agenda truly make the Summit a unique storage industry event.
Key End User and Vendor Presentations Similar to past Summits (we last convened in San Francisco in October of 2019) we enjoyed presentations from key end users including AWS, CERN, Meta/Facebook and Microsoft Azure. These end users are on the leading edge of innovation and in many ways are pioneering a path forward in the effective management of vast volumes of data growing exponentially every year.
From the vendor community, we were treated to the latest updates and soon to be unveiled products and solutions from Cloudian, IBM, Quantum, Spectra Logic, Twist Bioscience (DNA data storage) and Western Digital (HDD). The tape vendors shined a light on the continuing innovations in tape solutions including improvements in ease-of-use and maintenance of automated tape libraries as reviewed by Quantum. New tape applications abound from object storage on tape in support of hybrid cloud strategies as explained by Cloudian and Spectra, to the advantages of sustainable tape storage presented by IBM. It’s not a question of if, but when organizations will need to seriously address carbon emissions related to storage devices. After all: “no planet, no need for storage” quipped one attendee. Also included in the tape application discussions were the massive cold data archiving operations as presented by CERN and the hyper scale cloud service providers.
Finally from the world of tape, was a chilling, harrowing tale of a real life ransomware attack experienced by Spectra Logic and how their own tape products contributed to the safe protection of their data with the simple principal of a tape air gap.
Need for Archival Storage We also heard about the latest updates in the progress of DNA data storage from Twist Bioscience and where the world of HDD is going from Western Digital. We are now firmly in the zettabyte age with an expected 11 zettabytes of persistent data to be stored by 2025. Just one zettabyte would require 55 million 18TB HDDs or 55 million LTO-9 tapes. As an industry we are going to need a lot of archival storage! That includes future technologies like DNA, advanced HDDs, optical discs, and of course, highly advanced modern tape solutions. Tape will continue to deliver the lowest TCO, lowest energy consumption and excellent cybersecurity. All the while tape is supported by a roadmap with increasing cartridge capacities to meet market demand as it unfolds. Certainly, the cloud service providers will leverage all of these storage media at some point as they fine tune their SLAs and prices for serving hot data to cold archival data.
Fred Moore, Horison Information Strategies
Analysts Share Future Vision From the analyst community, we were treated to a visionary storage industry outlook from Fred Moore, president of Horison Information Strategies who shared the fact that 80% of all data quickly becomes archival and is best maintained in the lower tiers of his famed storage pyramid as an active archive or cold archive. Following Fred was important data from Brad Johns Consulting that showed the 18X sustainability advantage of eco-friendly tape systems compared to energy intensive HDDs. While we need both technologies, and they are indeed complementary, a tremendous opportunity exists for the storage industry to reduce carbon emissions by simply moving cold, inactive data from HDD to tape systems.
Rounding out the analyst presentations was Philippe Nicolas of Coldago Research with some valuable insights into end user storage requirements and preferences in both the U.S. and Europe.
Innovation from an Industry Expert From the realm of storage industry experts, we had a compelling talk from Jay Bartlett of Cozaint. With his expertise in the video surveillance market, Jay shared how the boom in video surveillance applications is becoming unsustainable from a retention of content perspective. It will become increasingly cost prohibitive to retain high definition video surveillance footage on defacto-standard HDD storage solutions. Jay revealed a breakthrough allowing for the seamless integration of tier 2 LTO tape with a cost savings benefit of 50%! No longer will we need to rely on grainy, compromised video evidence.
Final Thoughts The Summit wrapped up with a speaker panel moderated by IT writer and analyst, Philippe Nicolas. One big take away from this session was that while innovation is happening, it will need to continue in the future if we are to effectively store the zettabytes to come. Innovation means investment in R&D and production of new solutions, perhaps even hybrid models of existing technologies. That investment can’t come from the vendors alone and the hyper scalers will need to have some skin in the game.
In conclusion, the Summit was long overdue. The storage eco-system is a family from end users to vendors, to analysts and experts. As a family we learn from each other and help each other. That’s what families do. Fujifilm was pleased to bring the family together from around the globe under one roof, for frank and open discussion that will help solve the challenges we and our society are facing.
Reading Time: 2minutesBy Guest Blogger, Dr. Shawn O. Brume Sc. D., IBM Tape Evangelist and Strategist
According to a study by McKinsey, the average lifespan of companies listed in Standard & Poor’s is less than 18 years! That means that tape technology is already in business almost 4 times longer than the average S&P will survive. Tape technology celebrated 70 years young on May 21st. Tape has been and continues to be the most transforming data storage technology in history.
In the 50’s it was the only viable technology for storing data generated by the few computers in existence. In the 60’s tape took the world to the moon and preserved the data for usage nearly 40 years later when it was retrieved to assist in modern space explorations. By the 70’s Tape was dominating storage, transforming the financial industry by providing the ability to access data on accounts with minimal human intervention. The 80’s and 90’s continued the transformation of data availability by performing transactional data storage for ATMs, but also was key in the investigation of the space shuttle Challenger disaster; an investigation enhanced as a result of the durability of tape even when submerged in saltwater.
Today tape lives in the data center, preserving Zettabytes of data. Data being preserved and utilized across nearly every industry, examples:
Healthcare – Data preserved on tape is being utilized to develop new predictive health services. Digital medical records can be retained for the life of patients and shared across organizations.
Financial – Online transaction retention ensures customers valuable financial data is protected in the eventuality of a cyber-attack. Mortgage loans are preserved without fear of tampering.
Cloud – Data stored in public clouds are growing at a 30% faster rate than traditional storage. Cloud providers rely on tape to provide data durability and low-cost storage subscriptions.
Tape’s popularity has often been driven by the low cost of storage, modern data storage requires so much more including cyber-resiliency, data durability and low carbon footprints that enable sustainable IT.
Cyber Resiliency – Tape is the only true airgap data storage solution available.
Data Durability – Tape has a native single copy durability of 11- Nines. This means the likelihood of a single bit failure is 1 in 100 Petabytes.
Sustainability – At scale tape technology is 96% lower carbon footprint than highly dense HDD storage (when comparing OCP Bryce canyon and IBM tape technology with 27PB of data).
If preserving data, in a cyber-resilient solution, at low cost, with relatively low carbon impact meets your business outcomes, then why wait? Clearly tape is here to stay and surging in usage across nearly every business use case.
Happy 70-years to an amazing technology!
For more information about technology since tape’s introduction, check out this post from my colleague Mike Doran.
For more information on current tape products see the IBM product page.
The 2021 Active Archive Alliance annual market report has just been released, entitled “Saved by the Data. Active Archive Leads the Way in a Mid-Pandemic World”.
Certainly, the COVID pandemic was a shock to many companies and put tremendous strain on operations, revenue, and profit. But those companies who had already implemented a sensible active archive strategy were at a competitive advantage thanks to their ability to intelligently manage access to their data.
I think active archiving, the practice of keeping data online all the time and easily accessible to users, is a hot concept in storage right now because it is really about optimization – getting the right data in the right place, at the right time, and at the right cost.
We know that IT budgets are not keeping up with the relentless growth of data. We also know that 60% to 80% of data quickly becomes archival. Typically after 30, 60, or 90 days, files become static and the frequency of access drops off. So why keep that kind of data on expensive primary storage?
Why not let intelligent data management software that is typical of an active archive solution move that data by user-defined policy from high performance, expensive tiers, to lower performance but more cost-effective tiers like economy disk or tape systems, or even cloud? All while maintaining transparent access for users.
We know that the value of data is increasing, retention periods are getting longer, and users want to maintain ready access to their data without IT staff intervention. But we also need to worry about the bottom line, about efficiency, compliance, sustainability, and cybersecurity! Active archiving provides the right solutions to these worries and that’s why it is such a hot concept in storage today.
But enough said, read the full report here and check out what Alliance members had to say in their related virtual conference.
Usage of Cookies
Cookies are important to the proper functioning of a site. To improve your experience, we use cookies to remember log-in details and provide secure log-in, collect statistics to optimize site functionality and deliver content tailored to your interest. By continuing to use this site you are giving us your consent to do this. For more information you can read our Privacy Policy.
As both an active archive and tape evangelist, I’m excited to share how 
An essential active archive principle asserts that today’s enterprises need online access to legacy data. Some use cases may need fast access; in other instances, a longer retrieval time is acceptable. The organization determines its access needs and permissions for users and groups.
