Blog

The Active Archive Alliance recently released its 2020 Active Archive and the State of the Industry Report, which highlights the increased demand for new data management strategies as well as benefits and use cases for active archive solutions.

Here are a few takeaways from the report:

• Today’s data demands an intelligent archive solution that leverages the advanced capabilities of data movement technology and scale-out hardware to realize its fast-increasing value.
• The growing reliance on archival data makes it ground zero for unlocking game-changing data strategies.
• New applications, such as the Internet of Things (IoT) with billions of nodes and boosted by the arrival of 5G networks, will help fuel insatiable demand for more intelligent active archives.
• Key usage cases for active archiving tell the real story, starting with healthcare and high-performance computing (HPC) in life sciences.
• Other top use cases include security, business efficiency and continuity, media and entertainment, and IoT, including autonomous vehicles.

Sponsors of the report include Active Archive Alliance members Atempo, Fujifilm Recording Media USA, IBM, Iron Mountain, Harmony Healthcare IT, MediQuant, PoINT Software & Systems, Quantum, Qumulo, QStar Technologies, Spectra Logic, StrongBox Data Solutions and Western Digital.

You can read the full report here.

For more information on the Active Archive Alliance visit: www.activearchive.com.

Ransomware statistics can be frightening! Research studies suggest that over two million ransomware incidents occurred in 2019 with 60% of organizations surveyed experiencing a ransomware attack in the past year. To make matters worse, the cybercriminals have moved up the food chain. Two thirds of those attacked said the incident cost them $100,000 to $500,000. Another 20% said the price tag exceeded half a million. Overall, the losses are measured in billions of dollars per year. And it’s getting worse. Enterprise Strategy Group (ESG) reports that about half of all organizations have seen a rise in cyber attacks since the recent upsurge in people working from home.

Understandably, this is a big concern to the FBI. It has issued alerts about the dangers of ransomware. One of its primary recommendations to CEOs is the importance of backup with the following key questions:

“Do you backup all critical information? Are backups stored offline? Have you tested your ability to revert to backups during an incident?”

The key word in that line of questioning is “offline.” Hackers have gotten good at staging their attacks slowly over time. They infiltrate a system, quietly ensuring that backups are infected as well as operational systems. When ready, they encrypt the files and announce to the company that they are locked out of their files until the ransom is paid. Any attempt to recover data from disk or the cloud fails as the backup files are infected, too.

The answer is to make tape part of the 3-2-1 system: Three separate copies of data, stored on at least two different storage media with one copy off-site. This might mean, for example, one copy retained on onsite disk, another in the cloud, and one on tape; or one on onsite disk, one on onsite tape as well as tape copies stored offsite.

Read more

Part 1: What Are Hyperscale Data Centers?

Hyperscale data centers have spread across the globe to meet unprecedented data storage requirements. In this three-part blog series, we take a look at how the industry is preparing for the next wave of hyperscale storage challenges.

The term “hyper” means extreme or excess. While there isn’t a single, comprehensive definition for HSDCs, they are significantly larger facilities than a typical enterprise data center. The Synergy Research Group Report indicated there were 390 hyperscale data centersworldwideattheendof2017. An overwhelming majority of those facilities, 44%are in the US with China being a distant second with 8%. Currently the world’s largest data center facility has 1.1 million square feet. To put this into perspective the standard size for a professional soccer field is 60,000 square feet, the equivalent to about 18.3 soccer fields. Imagine needing binoculars to look out over an endless array of computer equipment in a single facility. Imagine paying the energy bill!

Hyperscale refers to a computer architecture that massively scales compute power, memory, a high-speed networking infrastructure, and storage resources typically serving millions of users with relatively few applications. While most enterprises can rely on out-of- the-box infrastructures from vendors, hyperscale companies must personalize nearly every aspect of their environment. A HSDC architecture is typically made up of tens of thousands of small, inexpensive, commodity component servers or nodes, providing massive compute, storage and networking capabilities. HSDCs are implementing Artificial Intelligence (AI), and Machine Learning (ML) to help manage the load and are exploiting the storage hierarchy including heavy tape usage for backup, archive, active archive and disaster recovery applications.

In Part 2 of this series, we’ll take a look at the characteristics of the hyperscale data center. For more information on this topic, download our white paper: The Ascent to Hyperscale.

By Rich Gadomski

I just spent a full day at a meeting of the Active Archive Alliance and as I was flying home it occurred to me that it’s time for data storage managers to rise up from the sleepy status quo of buying more disk arrays to address runaway data growth problems. It’s time to wake up and smell the sweet aroma of freshly made modern data tape (sort of like that new car smell if you don’t know).

Why do that you ask? Because best practices and undeniable facts say so. Consider the following:

Data goes through a lifecycle from hot to cold, that is to say from a period of active use to a period of inactivity. This can happen in as little as 30 days or less.

Inactive data should not stay on primary storage devices. It takes up space on expensive storage media, consumes more energy and adds to the backup burden.

What to do? Delete it? You probably can’t get permission to delete it, all data is now potentially valuable with new artificial intelligence (AI) and analytic tools emerging to derive value from that data. But you can move it and stop copying it!

Where do you move it to? Put it in an active archive consisting of low cost disk cache and even lower cost long term storage like a high density automated tape library. To store one petabyte of data for 10 years in a tape library will cost around $220,000 depending on your TCO variables. Alternatively, you could spend $900,000 on HDD and around $1,300,000 for cloud. Need more capacity? Tape libraries easily scale by adding more slots and tapes. You can export full tapes and plug new ones in. Move the full tapes offsite and get the benefit of air gap since the data is physically isolated from other networks. At least you know that data can’t be accessed and held for ransom.

Getting end user access requests for that data all of a sudden? Move it back to disk cache and serve it from there. When done, move it back to the tape library. Tape is super-fast, 360 MB a second and file access is made easier and faster with LTFS.

How to orchestrate all this? Intelligent data management solutions help move data automatically. Leverage metadata and AI tools to analyze files and move them off primary storage if they don’t belong there.

Does this sound like a tiered storage strategy? It is and it’s also known as an active archive. This is a best practice used by the biggest and most advanced data generating companies in the industry. If it works for them, it will work for you too.

There’s a lot of hype in the storage industry with lots of folks looking for new, better ways to do things. But some things are tried and true, like tape, with the benefits of constantly evolving capacities, performance, reliability and long term archivability. So wake up and smell the tape…put your data where it belongs and get on with your day!

Brookhaven National Labs (BNL) has grown from 60 PB of data archived in 2015 to 145 PB of data archived in 2018. In this Fujifilm Summit video, David Yu explains how BNL is using tape storage to cost-effectively manage this data growth. In addition, BNL uses an active archive system to provide easy access to data that is frequently needed by the BNL data center and other research institutions.

Explosive data growth continues to be a top challenge for today’s organizations and this growth is only going to increase in the future. In fact, according to analyst firm IDC, by 2025 worldwide data will grow 61% to 175 zettabytes, with as much of the data residing in the cloud as in data centers.

New technologies and approaches are continually being created to help address this data storage deluge. Members of the Active Archive Alliance from Fujifilm Recording Media, U.S.A., Inc, Spectra Logic, StrongBox Data and Quantum recently shared their insights into what the future looks like for active archives and data storage in 2019. Here are some of their top predictions:

Artificial Intelligence Creates Demand for Active Archives
The evolution of deep learning, machine learning and artificial intelligence will continue to expand in 2019 across every industry as the digital transformation wave produces an explosion of big data. With these AI tools, organizations will be able to extract more value from their data than ever before giving rise to an insatiable demand for more data, more analytics…more competitive advantage. A dramatic increase in storage and specifically active archive will be required to cost effectively and efficiently provide accessibility to big data at scale.

Flash Will Gain Wide-Scale Adoption, But a Need to Store Everything Will Make Secondary Storage More Important Than Ever
In the coming year we will see wide-scale adoption of flash storage. Organizations of all sizes will include solid-state drive (SSD) for greater performance, energy savings, space efficiency, and reduced management. New technologies like integrated data protection, storage federation/automation, policy-based provisioning, tiered data movement, and public cloud integration will be built on top of this flash foundation.

With the increased adoption of flash, organizations will also face the challenge of how to affordably store the data that is not mission critical, but still has value and therefore cannot be deleted. With the move to flash organizations will utilize a secondary storage tier to affordably manage all the organizations data, and this will happen through intelligent data management software designed to move data to a more affordable tier, without sacrificing access and searchability of the data.

Shift From Managing Your Storage to Managing Your Data
Data, not the underlying physical storage, is what matters. However, traditional storage systems are “big dumb buckets” that provide precious little insight into what data is growing, what applications or users are accessing it, or what is consuming storage performance and why.

Next-generation storage systems are “data-aware,” with real-time analytics built directly into the storage itself, delivering real-time information on data and performance at massive scale providing insight into data and storage.  As organizations better understand their data (how it is being generated, at what pace, by who, for what project) they are more informed as to how to plan and budget for the future growth of their data, and better understand how to move data to different tiers based on customized policies.

Cross-platform Storage Automation Reduces Costs, Increases Productivity
The reality is that there is not a “one-size-fits-all” storage solution that addresses the multiple requirements faced by most organizations.  The result is that large environments typically rely on multiple storage vendors and point solutions to address the different performance and cost profiles needed for their data. The problem is this adds complexity for IT managers, requiring them to do more with static or shrinking operational budgets. This trend is driving a demand in the industry for solutions that provide automation of data and storage resource management across any storage type from any vendor. Such solutions leverage policy engines and management tools that are driven by multiple types of metadata about the files and their business value as they evolve over time. Such automation tools help data managers know what they have, and gives them control of cross-platform data migration, tiering, active archiving, and protection, without interrupting users. This type of metadata-driven automation will be an increasing trend over the next few years, because it provides demonstrable ROI by reducing OPEX and complexity for IT, breaking storage vendor lock-in, while increasing storage utilization efficiency and user productivity.

Rich Media Content Will Grow Exponentially, Across Many Industries
Video now constitutes 50% of all data. Rich media comprises our video surveillance; consumer images, voice and video; medical imagery, IoT, entertainment and social media. Large and unstructured data is often 50 times or larger than the average corporate database. Video is unique, and it is not typically a good fit for traditional backup; it cannot be compressed or deduplicated, it doesn’t work well with replication, snaps or clones, and ingest speed is critical. Rich media is projected to surpass 100 Zetabytes worldwide by 2020. Expect enterprise data services to be increasingly optimized for large or rich media data sets, with infrastructure optimized for ingest processing and the full life cycle management of forms of rich media.

By Rich Gadomski

At Storage Visions 2018, held in Santa Clara this past October, I had the opportunity to talk about the future outlook for tape as attendees wanted to know how they were going to store all the data that’s being created. The session I participated in was entitled “Epic Battles with Classic Heros – Flash, HDDs and Tape Slay Data Challenges.” As the title suggests, battling exponential data growth takes more than one storage media type to effectively handle the deluge of data that’s being created (now estimated to be 33 ZB in 2018 and growing to 175 ZB by 2025, according to IDC).

As our session moderator, Jean Bozeman from Hurwitz & Associates pointed out in her introduction, a variety of storage workloads create the need for a spectrum of storage technologies. Certainly the need for speed at the top of the storage pyramid is being addressed by performance HDD and increasingly by ever evolving solid state drives.

The need for longer term storage at scale is the domain of capacity HDD and of course, tape. Managing the data deluge is all about having the right data on the right storage medium at the right time. Not everything can or should be stored on expensive high performance flash. You need high capacity optimized media for long term data retention and that’s where HDD and tape come in to play (often in a user friendly active archive environment).

When it comes to the future of capacity in the domain of HDD, current perpendicular magnetic recording technology has reached  ‘super paramagnetic” limitations where increasing areal density to increase capacity is not a viable option. With helium filled HDDs, more platters can fit in the same form factor as air filled HDDs but this has not allowed a significant growth in capacity.  New technology concepts such as Heat Assisted Magnetic Recording (HAMR) and Microwave Assisted Magnetic Recording (MAMR) are on the horizon but market availability has been delayed. There is also the potential of vacuum sealed HDDs with better operating characteristics than helium that could help HAMR and MAMR HDDs get up to 40 – 50 TB at some point in the future.

But fundamentally, increasing capacity of a storage medium and ultimately reducing its cost is best achieved by increasing areal density. This is where magnetic tape technology really shines as today’s modern tape with per cartridge capacities already as high as 20 TB having very low areal densities compared to HDD.

Therefore, tape has a long runway before facing areal density limits and as a result, future tape roadmaps have the potential to achieve up to 220 TB on a standard form factor cartridge using Barium Ferrite (BaFe) magnetic particles and up to 400 TB using next generation Strontium Ferrite (SrFe). At the same time, both BaFe and SrFe can maintain magnetic signal strength integrity for at least 30 years making them ideal not only for high capacity but for cost effective long term data retention as well.

“No wonder the cloud guys are all using tape now,” exclaimed an attendee in the audience during the Q&A. They certainly also use a lot of flash and a lot of disk too. It is an epic battle and it takes a spectrum of storage technologies to slay the data challenges.

Since its launch in March 2002, the YES Network has been the number 1 rated regional sports network in the USA. From its inception, YES has striven for the highest-quality picture and sound and to be at the leading-edge of broadcast technology. To this end, YES was constructed as an all-digital facility.

To manage its growing library, the network launched a digital archive project. Initially, the plan was to find a way to convert HD content into a file format that could be stored in a system so that producers and directors could easily find and retrieve selected plays to be integrated into classic game and other shoulder programmes. Avid had provided the YES editing systems from the outset, and the original five Avid editing systems were connected to an Avid Omega JBOD array for storage.

This paper provides a deep dive into the pros and cons of local, cloud, solid-state and linear tape-open storage solutions. It opens with YES Network Director of Engineering and Technology John McKenna’s account of the YES Network’s digital transformation, and is followed by YES Network Engineering Department Project Manager of Broadcast Systems Jason Marshall’s summary of modular to virtual technology migration. This paper details ratios on high-performance broadcast data systems, as well as power consumption and solution trade-offs. This paper aims to gain the reader’s confidence in virtualising a media asset system as well as converting linear systems to packet-based media technologies including transcoding, Active Archive and future SMPTE 2110 solutions.

Read the full paper here: Migrating to a Virtual Environment