FUJIFILM INSIGHTS BLOG

Archival data is piling up faster than ever as organizations are quickly learning the value of analyzing vast amounts of previously untapped digital data. Industry studies consistently find that the vast majority of all digital data is rarely, if ever, accessed again after it is stored. However, this is changing now with the emergence of big data analytics made possible by Machine Learning (ML) and Artificial Intelligence (AI) tools that bring data back to life and tap its enormous value for improved efficiency and competitive advantage.

The need to securely store, search for, retrieve and analyze massive volumes of archival content is fueling new and more effective advancements in archive solutions. These trends are further compounded as an increasing number of businesses are approaching hyperscale levels with significant archival capacity requirements.

An active archive resolves complexity by leveraging the benefits of an intelligent data management layer, and an increasing number of effective software products that address this are now available. Access and management of data is getting more complex and is requiring modern strategies with intelligent data management techniques. These strategies and techniques are now being enhanced by AI to further improve and automate data management.

Download the Active Archive Alliance 2019 State of Active Archive Report to learn more about the state of the archive market and the expanding role the active archive plays.

By Rich Gadomski

I just spent a full day at a meeting of the Active Archive Alliance and as I was flying home it occurred to me that it’s time for data storage managers to rise up from the sleepy status quo of buying more disk arrays to address runaway data growth problems. It’s time to wake up and smell the sweet aroma of freshly made modern data tape (sort of like that new car smell if you don’t know).

Why do that you ask? Because best practices and undeniable facts say so. Consider the following:

Data goes through a lifecycle from hot to cold, that is to say from a period of active use to a period of inactivity. This can happen in as little as 30 days or less.

Inactive data should not stay on primary storage devices. It takes up space on expensive storage media, consumes more energy and adds to the backup burden.

What to do? Delete it? You probably can’t get permission to delete it, all data is now potentially valuable with new artificial intelligence (AI) and analytic tools emerging to derive value from that data. But you can move it and stop copying it!

Where do you move it to? Put it in an active archive consisting of low cost disk cache and even lower cost long term storage like a high density automated tape library. To store one petabyte of data for 10 years in a tape library will cost around $220,000 depending on your TCO variables. Alternatively, you could spend $900,000 on HDD and around $1,300,000 for cloud. Need more capacity? Tape libraries easily scale by adding more slots and tapes. You can export full tapes and plug new ones in. Move the full tapes offsite and get the benefit of air gap since the data is physically isolated from other networks. At least you know that data can’t be accessed and held for ransom.

Getting end user access requests for that data all of a sudden? Move it back to disk cache and serve it from there. When done, move it back to the tape library. Tape is super-fast, 360 MB a second and file access is made easier and faster with LTFS.

How to orchestrate all this? Intelligent data management solutions help move data automatically. Leverage metadata and AI tools to analyze files and move them off primary storage if they don’t belong there.

Does this sound like a tiered storage strategy? It is and it’s also known as an active archive. This is a best practice used by the biggest and most advanced data generating companies in the industry. If it works for them, it will work for you too.

There’s a lot of hype in the storage industry with lots of folks looking for new, better ways to do things. But some things are tried and true, like tape, with the benefits of constantly evolving capacities, performance, reliability and long term archivability. So wake up and smell the tape…put your data where it belongs and get on with your day!

Kevin Benitez
Product Marketing Manager
FUJIFILM Recording Media U.S.A., Inc.

Over the past decade, the Media and Entertainment (M&E) industry has experienced a considerable increase in the amount of data generated due to the transition from traditional media workflows to fully digital environments. Today, the retention and accessibility of digital assets and video are incredibly vital to maintaining a competitive advantage. Fujifilm understands M&E companies’ digital storage challenges; that’s why companies like MLB, The LA Kings Ice Hockey Team, Chainsaw Edit, and others have turned to Fujifilm tape to ensure the integrity of their video assets while drastically reducing long-term storage costs.

Today, tape storage is used in modern infrastructure to deliver high storage capacities with low cost of ownership compared to other storage solutions.

Modern M&E companies continue incorporating data tape storage into their environments to combat costs. Nowadays storage requirements are on a different scale from where M&E companies first started, and data will continue to increase exponentially as the industry moves from HD to 4K, and soon to 8K recording. Today, a single digital 4K camera can record up to 1.5 TB for every hour of filming. Before this, companies like the LA Kings only recorded 200 GB in an entire year. Most IT budgets in the industry are not growing enough to support today’s data deluge—of which data storage can consume 70%.

“Tape storage in general is the lowest cost storage of any type of storage,” says Storage Economics expert Brad Johns

The cost of using LTO-7 tape is as low as $0.01/GB which can be 7 times less expensive than disk storage over a ten year period.* Additionally, tape doesn’t use any energy when it’s not being used, on the other hand, disk systems use 76 times more electrical power than a similarly configured tape system.* “Tape storage in general is the lowest cost storage of any type of storage,” says Brad Johns, founder of Brad Johns Consulting LLC, a storage consulting firm.

Why are leading M&E companies turning to tape?

• Extremely cost-effective
• Highly reliable
• Portable to use at remote locations
• Scalable to extremely large capacities
• Open standards to allow the interchange of files

LTO tape is an ideal solution for M&E companies. LTO is an open format designed for interoperability and together with LTFS, provides easy data access and management—perfect for easy file share, high performance, and improved workflow.

How are M&E companies are using tape today?

NewBay Media has compiled how LTO technology and LTFS manage every stage of content creation/ management:

1. Production. LTO technology with LTFS protects original content with on-site backup copy, reduces camera media inventory costs, and enables the interchange of content between production sites and post-production.
2. Post-Production. LTO technology with LTFS offers a low-cost storage solution for work-in-progress, scales to meet large capacities, provides a standard means of interchange across the post-production ecosystem, and gives users the ability to offload less active content from expensive, high-performance flash or disks.
3. Distribution. LTO technology with LTFS supports the transfer of large amounts of digital content at low cost and serves as the de-facto standard for submission of content—to studios and between business partners.
4. Archiving. LTO technology with LTFS is ideal for long-term storage due to its durability, reliability, and low cost of operation. It scales to meet very large capacity requirements and supports rapid restoration for the repurposing of content.

Tape’s low-cost acquisition price per gigabyte and TCO advantage compared with other storage mediums make tape the most cost-effective technology for long-term data retention. Today, M& E companies can compare the total cost of ownership of data retention using TCO calculators created by Brad Johns Consulting, https://page.dternity.net/TCO.html, and the LTO consortium, https://www.lto.org/resources/.

More M&E companies are seeing the advantages of LTO tape, which can store massive amounts of data and combat ever-increasing storage costs across production, post-production, distribution, or archiving.

*Source: ESG report “Analyzing the Economic Value of LTO Tape for Long-Term Data Retention.”

*Source: The Clipper Group “Continuing the Search for the Right Mix of Long-Term Storage Infrastructure —A TCO Analysis of Disk and Tape Solutions.”

In this video, Fred Moore, president of Horison Information Strategies explains the benefits of an active archive including improving tape performance, providing random access to tape data, and eliminating mount time. Watch the video here for more information:

By Rich Gadomski
V.P. Marketing
FUJIFILM Recording Media U.S.A., Inc.

Have you ever noticed the grainy quality of surveillance footage shown on the local news broadcasts? Some unfortunate citizen is getting assaulted after making an ATM withdrawal and you feel like you’re watching a 1930s Charlie Chaplin rerun. The reason for this is that most organizations charged with surveilling public places can’t afford to do so in high resolution and certainly can’t afford to keep the surveillance content for any considerable period of time. Yet wouldn’t it be nice to easily ID the suspect and maybe go back to other video databases to see if there’s a pattern occurring over time?

As a society we want better security and swifter justice. We have the technology to achieve this but we don’t necessarily have the budget. The good news is that a record number of surveillance cameras are shipping year after year. They are also becoming more affordable and resolutions are steadily increasing to include 4K or better.

In fact, demand for video surveillance systems is continuing to explode driven by several factors including increased security threats, legislation, IoT applications, law enforcement applications and increasing affordability of surveillance cameras.

All of this is taking place at a time when society is demanding more transparency into what is happening in our public places and in law enforcement actions. This has resulted in a steady increase in evidentiary content from facility security cameras to body worn cameras, dashboard cameras, interview room audio and video recordings, cell phone data and others. This proliferation of surveillance technology has given rise to the “transparency paradox” where the public demands more recorded evidence which creates the need for more data storage. But we only have limited budgets that can’t handle the increasing data retention burdens.

This is where the concept of an active archive comes into focus providing a solution where organizations can afford to maintain online access to all of their surveillance content in a multi-tiered storage system that leverages the speed of flash and disk with the superior economics of tape. In a typical active archive environment, the file system extends across all storage tiers from primary flash/disk to long term, economy tape storage. Content moves by policy from high cost primary storage to low cost, long term storage. Typically this means LTFS LTO tape featuring the highest capacity, longest archival life, highest reliability and lowest total cost of ownership.

With an active archive in place, organizations can easily and cost effectively maintain more surveillance content at higher resolutions for longer retention periods and can maintain this on-premises to safely control chain of custody.

Expect demand for active archives to solve the “transparency paradox.” It should make the local news more engaging too!