Blog

Archival data is piling up faster than ever as organizations are quickly learning the value of analyzing vast amounts of previously untapped digital data. Industry studies consistently find that the vast majority of all digital data is rarely, if ever, accessed again after it is stored. However, this is changing now with the emergence of big data analytics made possible by Machine Learning (ML) and Artificial Intelligence (AI) tools that bring data back to life and tap its enormous value for improved efficiency and competitive advantage.

The need to securely store, search for, retrieve and analyze massive volumes of archival content is fueling new and more effective advancements in archive solutions. These trends are further compounded as an increasing number of businesses are approaching hyperscale levels with significant archival capacity requirements.

An active archive resolves complexity by leveraging the benefits of an intelligent data management layer, and an increasing number of effective software products that address this are now available. Access and management of data is getting more complex and is requiring modern strategies with intelligent data management techniques. These strategies and techniques are now being enhanced by AI to further improve and automate data management.

Download the Active Archive Alliance 2019 State of Active Archive Report to learn more about the state of the archive market and the expanding role the active archive plays.

When a storage array goes down, or a required file is unavailable, your business cannot take a vacation. Users must have access to a secondary copy of that file to ensure the business keeps running. This might involve accessing a version of the file or of metadata that was written a minute ago, a week ago, or to restore a replicated off-site copy.

In this video, Floyd Christofferson, CEO of Strongbox Data Solutions, explains how StrongLink allows users to establish disaster recovery workflows, and to automatically access secondary copies on another array, or in the cloud, or to pull from a file archived on tape. StrongLink automatically manages, monitors and replicates data by policy, ensuring the right number of copies are maintained across the correct storage locations (file & object stores, cloud or archive). Data is then monitored for long-term data preservation to meet compliance and governance requirements.

Read more here:

Most storage environments consist of multiple platforms, vendors, and clouds creating a unique set of problems: lack of data visibility to enable tiering that aligns data value & storage costs, data locked in silos each with their own management tools, and complicated data protection.

In this video, Floyd Christofferson, CEO of Strongbox Data Solutions, explains how StrongLink Autonomous Engine™ with Data Insights and analytics quickly, dynamically, & seamlessly automates data movement and storage tiering across platforms, vendors and cloud. Data is managed in real-time to facilitate workflows & collaboration, SLAs and StrongLink Autonomous Engine QoS, while eliminating resource contention issues.

Read more here:

Get the Most Out of Your Isilon with Autonomous Data Management.

When an Isilon cluster fills up, IT is faced with deciding whether to buy more Isilon, move data to another platform, or archive/tier data off the Isilon. The challenge is figuring out how to maximize the value of the existing storage investments and reduce storage costs.

What if you could connect Isilon with other storage types into a cross-platform global namespace while reducing costs and improving business continuity and disaster recovery?

Stronglink automatically migrates data from Isilon to other storage platforms—including tape and public cloud—without interrupting user access and while maximizing the value of your existing storage investments.

Read more here: Solution Brief

Kevin Benitez
Product Marketing Manager
FUJIFILM Recording Media U.S.A., Inc.

Over the past decade, the Media and Entertainment (M&E) industry has experienced a considerable increase in the amount of data generated due to the transition from traditional media workflows to fully digital environments. Today, the retention and accessibility of digital assets and video are incredibly vital to maintaining a competitive advantage. Fujifilm understands M&E companies’ digital storage challenges; that’s why companies like MLB, The LA Kings Ice Hockey Team, Chainsaw Edit, and others have turned to Fujifilm tape to ensure the integrity of their video assets while drastically reducing long-term storage costs.

Today, tape storage is used in modern infrastructure to deliver high storage capacities with low cost of ownership compared to other storage solutions.

Modern M&E companies continue incorporating data tape storage into their environments to combat costs. Nowadays storage requirements are on a different scale from where M&E companies first started, and data will continue to increase exponentially as the industry moves from HD to 4K, and soon to 8K recording. Today, a single digital 4K camera can record up to 1.5 TB for every hour of filming. Before this, companies like the LA Kings only recorded 200 GB in an entire year. Most IT budgets in the industry are not growing enough to support today’s data deluge—of which data storage can consume 70%.

“Tape storage in general is the lowest cost storage of any type of storage,” says Storage Economics expert Brad Johns

The cost of using LTO-7 tape is as low as $0.01/GB which can be 7 times less expensive than disk storage over a ten year period.* Additionally, tape doesn’t use any energy when it’s not being used, on the other hand, disk systems use 76 times more electrical power than a similarly configured tape system.* “Tape storage in general is the lowest cost storage of any type of storage,” says Brad Johns, founder of Brad Johns Consulting LLC, a storage consulting firm.

Why are leading M&E companies turning to tape?

• Extremely cost-effective
• Highly reliable
• Portable to use at remote locations
• Scalable to extremely large capacities
• Open standards to allow the interchange of files

LTO tape is an ideal solution for M&E companies. LTO is an open format designed for interoperability and together with LTFS, provides easy data access and management—perfect for easy file share, high performance, and improved workflow.

How are M&E companies are using tape today?

NewBay Media has compiled how LTO technology and LTFS manage every stage of content creation/ management:

1. Production. LTO technology with LTFS protects original content with on-site backup copy, reduces camera media inventory costs, and enables the interchange of content between production sites and post-production.
2. Post-Production. LTO technology with LTFS offers a low-cost storage solution for work-in-progress, scales to meet large capacities, provides a standard means of interchange across the post-production ecosystem, and gives users the ability to offload less active content from expensive, high-performance flash or disks.
3. Distribution. LTO technology with LTFS supports the transfer of large amounts of digital content at low cost and serves as the de-facto standard for submission of content—to studios and between business partners.
4. Archiving. LTO technology with LTFS is ideal for long-term storage due to its durability, reliability, and low cost of operation. It scales to meet very large capacity requirements and supports rapid restoration for the repurposing of content.

Tape’s low-cost acquisition price per gigabyte and TCO advantage compared with other storage mediums make tape the most cost-effective technology for long-term data retention. Today, M& E companies can compare the total cost of ownership of data retention using TCO calculators created by Brad Johns Consulting, https://page.dternity.net/TCO.html, and the LTO consortium, https://www.lto.org/resources/.

More M&E companies are seeing the advantages of LTO tape, which can store massive amounts of data and combat ever-increasing storage costs across production, post-production, distribution, or archiving.

*Source: ESG report “Analyzing the Economic Value of LTO Tape for Long-Term Data Retention.”

*Source: The Clipper Group “Continuing the Search for the Right Mix of Long-Term Storage Infrastructure —A TCO Analysis of Disk and Tape Solutions.”

By: Fred Moore, President
Horison Information Strategies
www.horison.com

Introduction

The traditional storage market is shifting as applications are more effectively exploiting the tiered storage hierarchy to better align availability requirements, service levels, and data protection mandates with the optimal infrastructure cost. Clearly HDDs remain and for the foreseeable future will continue to be the work-horse of the storage hierarchy. They are steadily losing market share for response time critical, high performance applications to the growing deployment of SSD technology while losing many lower activity, archival and resilience applications to significantly improved modern tape technology. The pressure is on the HDD industry and is illustrated by worldwide HDD shipments (data from Statista), which peaked with 651,300 million in 2010 and dropped 35% to 403,710 million in 2017. HDD shipments are predicted to fall to 341,950 million in 2020. Data which in prior years was often stored on HDDs without much thought to storage optimization is now taking up residence elsewhere. As storage pools get larger, the need to optimize storage by getting the right data in the right place also gets larger.

What’s Behind the Shift?

SSDs mean high performance. SSDs have successfully addressed much of the high-performance storage market that was basically the exclusive domain of HDDs. Within the next 12-18 months, solid-state flash arrays currently using 2D NAND are projected to improve in performance by a factor of 10x and double in density and cost-effectiveness as 3D NAND and 3D XPoint technology begins to emerge. This technological progression will significantly change the dynamics of the performance centric storage market. Compared to HDDs, SSDs have higher data-transfer rates, faster access times, better reliability, much lower latency with lower energy consumption. For most users, the consistent and high speed at which SSDs can read and write data and meet service levels is the key attraction. Because SSDs have no moving parts, they can operate at speeds far above those of a typical HDD. Fragmentation is not an issue for SSDs. Files can be written anywhere with little impact on R/W times, resulting in read times far faster than any HDD.

HDDs can handle every data type and have carried the most of load for the storage industry for years, however future challenges for HDDs are mounting. HDDs are increasing in capacity but not in performance as the IOPS (I/Os per Second) for HDDs have basically leveled off. The potential for more concurrently active data sets or files increases as HDD capacity grows and the increased contention for the single actuator arm causes erratic response time delays. Excessive RAID rebuild times are a growing concern and it can now take several days to rebuild a failed HDD in a RAID array degrading performance during the lengthy rebuild period. As HDD capacities continue to increase, total time required for the RAID rebuilding process will become prohibitive for many IT organizations and higher capacity HDDs could force a replacement for traditional RAID architecture implementations. HDD areal density is currently progressing at ~16% annually, about half the rate of tape technology. HDD capacity is often increased by adding more platters as the available surface recording are is squeezed as areal density increases. HDDs have a much higher TCO and use considerably more energy than tape or SSD.

For tape, significant technology improvements over the past 10 years have resulted in a tape renaissance. These changes enable tape to provide the lowest acquisition cost and TCO, the highest capacity, fastest data transfer rates, lowest energy consumption and most reliable storage medium available. Tape reliability has surpassed that of HDDs by three orders of magnitude. Over the last 10 years, LTO tape has increased capacity 1,400%, performance 200%, and reliability 9,900% while modern tape media life now exceeds 30 years. Tape data rates are now nearly 2x faster than HDDs and are projected to be 5x faster by 2025. New features like the Active Archive, RAIT and RAO add significant performance and access time improvements beyond traditional tape. Using tape for cloud archives, rather than HDDs, greatly reduces cloud TCO and creates a “green cloud”. The steady innovation, compelling value proposition and new architectural developments demonstrate tape technology is not sitting still and the renaissance is expected to continue indefinitely.

Summary

A fundamental shift in the storage landscape is well underway as high-performance data moves from HDDs onto flash SSD while lower activity, resiliency and archive data migrate from HDD to modern tape. For the foreseeable future, HDDs will remain the home for many primary storage, mission- critical data along with the highest availability applications, but HDD shipment growth rates have declined nearly 35% since its highpoint in 2013 and projections indicate no signs of ending. As SSDs and tape continue to show rapid improvements and re-balance

the traditional tiered storage hierarchy, HDDs will continue to feel more pressure. The storage squeeze play is underway, and HDDs are caught in the middle.

Storytelling is a central facet of society that may have changed formats over the years but will never become obsolete. In today’s digital world, broadcasters and television networks focus on creating relatable stories to connect with their audiences, and they can’t do that without a wealth of readily available content.

MLB Network is the source for baseball stories of all kinds, from live games to studio shows and feature programming. Launched on January 1st, 2009, MLB Network is growing fast, reaching more than 70 million households today, delivering the best of America’s national pastime, all the time.

The Challenge: Digital Content Storage and Management

“MLB Network’s goal is to bring baseball to our audience every night with the highest levels of production quality, focus and enthusiasm throughout the year,” said Tab Butler, Director of Post Production and Media Management at MLB Network. “To accomplish this, we need constant access to our archives and current live game content. We need all information from every game securely stored and easily accessible.”

With multiple recordings of every game, along with multiple audio sources, and pregame, post-game and isolated camera feeds, it is not uncommon for MLB Network to record more than 3,000 hours of content per week. That content is then categorized and cataloged for future use, using the Emmy Award-nominated media asset management DIAMOND System. When the baseball season comes to a close, MLB Network continues to deliver baseball news 24/7, with special programming about a team, player or other happenings in the sport. These individual projects require systematic archival that supports precise selection and instant access of specific files. The challenge is how to empower diverse departments to directly access their projects, without heavy IT support.

MLB Network Deployed StrongBox for Project-Based Workflow

As an early adopter, MLB Network deployed a custom-developed StrongBox to manage archived projects in late 2011. StrongBox is a vendor-neutral, fully portable data vault for long-term file retention. Functioning as standard network-attached storage (NAS), StrongBox employs Linear Tape File System (LTFS) media as the principal storage medium to save money and empower a file-system view of all archived content. An internal disk cache enables rapid file access. With drag-and-drop functionality, StrongBox makes accessing archived projects easy for MLB Network, delivering content on-demand to multiple, simultaneous users. Since MLB network is a 24/7 operation, the production staff uses a SAN-based Final Cut Pro (FCP) platform to develop programming that is updated throughout the season with the latest information. When these projects are ready for archival, the video, audio and revision files, along with their metadata, are stored in StrongBox. Having the ability to recall an archived show, and repackage it with current information, utilizing tape as the storage medium, reduces the storage costs for the archived content. “StrongBox has been natural for streamlining this type of project-based storage. Instead of keeping projects on spinning disk, we’re able to offload to StrongBox,” explained Butler. One of Butler’s key initiatives is finding ways to better automate the media management environment, allowing different departments to manage their own archival data instead of relying on his Media Management team to store and retrieve files. With StrongBox, editors have direct access to archived projects in real-time, without having to depend on a Media Management operator to retrieve files.

High-Capacity, Low-Cost LTFS Tape for Proxy

Butler said that for the 2009 baseball season, 25 terabytes of spinning disk storage was required for the video proxy data, and for the 2010 baseball season, 32 terabytes was required. Although this proxy information requires long-term storage, it is accessed very infrequently. Thus, the operational costs for keeping this much data on spinning disk become extremely expensive. Even though it is long-tail content, it cannot be taken offline.

Through the custom-built, DIAMOND System, MLB Network logs and categorizes HD recording by viewing the video using proxy video files, which are recorded in real time. The bulk of recorded HD Video content is stored within an LTO library, and is searched and accessed using DIAMOND and the Grass Valley Aurora systems. Thus, Butler is investigating ways to use tape to further drive down the costs of his long-tail proxy content which is currently on spinning disk.

“If I look down my future roadmap, my proxy environment is going to continue to grow year over year for the lifetime of the archive,” explained Butler. “Getting the proxy on LTO-5 media is much more cost-efficient for long-tail content.” With the introduction of LTFS, tape can be partitioned and indexed on a file level. This brings significant opportunity for media and broadcast companies. For MLB Network, LTFS brings the capability to efficiently maintain an accessible archive at the file level, while eliminating the heat generation, cooling requirements, spinning drives and other operational costs associated with disk storage.

“StrongBox is a very unique product, with flexibility that makes it functional in multiple use-cases for MLB Network,” Butler continued. Currently, the MLB Network video archive consists of 10 to 12 petabytes of stored HD content, and 275 terabytes of proxy content. A migration of that much data would be a significant undertaking. With the ability to scale up to 35 petabytes, StrongBox delivers low-cost, high-capacity storage with high-performance access that could provide a cornerstone in the foundation for MLB Network’s biggest business asset – its programming.

The Bottom Line

MLB Network delivers exciting, engaging baseball stories 24/7, 365 days a year. With a massive content archive that will only continue to grow, Tab Butler knows that LTFS tape is a cost-efficient and scalable way to manage MLB Network’s digital records. While editors constantly juggle multiple projects with demanding deadlines, StrongBox facilitates a project-based workflow, integrating with the editing environment to allow end users more direct access to their archived content. Ultimately, StrongBox helps MLB Network spend more time creating and delivering award-winning baseball stories and less time worrying about how to manage data.

MEET THE FASTEST LTFS NAS

David Marglous
Inside Sales Representative
FUJIFILM Recording Media U.S.A., Inc

For over 50 years Fujifilm has been driving big impacts for customers and communities. One of our primary corporate goals is to enhance the quality of life for people worldwide, ranging from sustainable forestry to leveraging biodegradable resources. Whether you are a dealer ordering for a customer or a customer requesting Fujifilm media, we are here to help businesses save energy, money, and the environment.

At our Bedford, Massachusetts manufacturing facility we have installed 1,870 solar panels to promote renewable energy, eliminating 1,446 tons of CO2 emissions to date. Also, Fujifilm has moved to DPET plastic for P-cases and Library packs which requires 65% less energy to produce and has the lowest carbon footprint in the market. We understand that we too have a role in preserving our forests which is why we only use shipping pallets that are accredited by the Forest Stewardship Council (FSC C112029). Additionally, we are proud to be using Soy Ink which is biodegradable and renewable.

Just the other day I received a call from a long-time customer who was struggling with limited space and disposal issues. Determined to find an easier solution the customer asked: “what do we do with all the unneeded P-cases and insert cards?” To my delight, I replied “have you tried our library packs that hold 20 LTO cartridges or 50 corrugated pack cartons that hold ten 5-pack bricks of LTO without P-cases? This would eliminate your stress and reduce waste!”

How can you too become more socially responsible?

Next time you order, instead of placing your regular order for 100 LTO-7’s consider ordering five library packs or two 50 packs (as always, barcoding is entirely free). Not only will the new packaging reduce library load time by 75% but it will also eliminate 80% of waste when compared to the standard P-case.

For those of you looking to reduce waste and be more environmentally responsible, consider library packs and 50 packs next time you order media from Fujifilm. To learn more about our commitment to social responsibility, visit our sustainability website.

Need help making the best decision for your situation? Reach out directly to me!  dmarglous@fujifilm.com

Kevin Benitez
Product Marketing Manager
FUJIFILM Recording Media U.S.A., Inc.

We know your data is important, but let’s face it, data growth is rising exponentially, and there is no turning back. No matter the industry, the daily creation of data is mindboggling, whether it’s emails, videos, IoT, blockchain or something new, it doesn’t matter because for every keystroke typed there is a new set of data created, computed, and stored forever. On the bright side, if you are using LTO-5 tape for backup/archive, now is the perfect time to migrate to LTO-7 since it could save your IT department $1,276,280 over the next five years.

There are significant benefits to migrating from LTO-5 to LTO-7. If the impressive transfer rate of 300 MB / sec or enormous 6 TB capacity of LTO-7 hasn’t won you over, surely the cost savings will. IT budgets are only growing at an average of 7% annually. Meanwhile, the average data growth is between 35% and 65%, compounding yearly! In order for IT departments to meet future budget requirements, they need to unlock the economic value of LTO-7.

A recent cost-benefit analysis conducted by Brad Johns Consulting discovered that migrating from LTO-5 to LTO-7 consistently generated greater TCO savings across all storage capacities if the customer’s data was growing at all.

Brad Johns’ study investigated TCO over a five-year period for LTO-5 and LTO-7 for three different capacities 500TB, 2500TB, 5000TB and annual growth rates ranging from 10% to 50%. IT departments with the annual growth rate of 40% can expect savings from $135,308 to $1,276,280 over the next five years by switching to LTO-7.

Methodology

• Only new media and drives were purchased.
• The LTO-5 configurations were configured with a minimum of two tape drives and 100 tape cartridges per drive.
• The number of LTO-7 drives in the new configuration provided at least the same aggregate data rate as the LTO-5 base configuration (also with a two-drive minimum).

Overall the math shows migrating from LTO-5 to LTO-7 can significantly reduce cost, especially if a business has the ability to install LTO-7 drives in existing libraries rather than requiring a new library.

Not only do LTO-5 customers have a significant financial incentive to migrate to LTO-7, but they have additional technical benefits. The LTO-7 tape drives can read LTO-5 media as well as reading and writing on LTO-6 media. If LTO technology, in general is a concern, you can be confident that LTO is here to stay, and future proofing is already in development with Generation 12 and beyond. Bottom line, LTO-7 definitely has a place in your archive for long-term preservation management, especially if you are currently using LTO-5.

Have questions or need help making the decision?

Contact Us

The University of Notre Dame, based in South Bend, Indiana is a world class university with students that represent all 50 states and over 100 countries. Rigorous academics, NCAA Division I athletics, and numerous events and activities create an environment in which students are ignited with a passion to learn and to live to their fullest capacity.

Consistently ranked in the Top 25 institutions of higher learning by US News & World Report, Notre Dame has over 10,000 students, 4 colleges, 67 undergraduate programs, more than 50 graduate programs, and is home to the famous Fighting Irish Football team that draws fans from all around the world.

The Challenge

“The mission of the Office of IT is to serve as the trusted IT partner to all the Faculty, Staff and Students” says Mike Anderson, Storage Engineer at the University of Notre Dame. “It can be very difficult to satisfy their increasing demands for storage”. Historically IT was responsible for providing enterprise class storage for students, faculty and staff. This was primarily for user accounts, database and email. Large file storage was discouraged and requests for such storage were often denied. “Sometimes we would get requests from a department to store 100 TB or more” says Mike Anderson. “We simply did not have a solution that could accommodate a large archive at that point”.

“We needed an affordable storage solution that could scale to beyond a petabyte and provide data protection to all archived content”
– Mike Anderson, Storage Engineer at the University of Notre Dame

By 2014 it was clear to IT that something had to change. Requests for large file storage was growing and so was the need to archive them. Primary sources of this big data were the Athletics Department that was using more and more cameras to film all their games. Another source was the ever growing University Archives that was making an official effort to digitize and preserve historical data for future use. “We needed an affordable storage solution that could scale to beyond a petabyte and provide data protection to all archived content” said Mike Anderson.

The Solution

“After evaluating many solutions we decided to go with the Dternity NAS and Media Cloud replication service”. The solution is built to fit into existing environments and deliver the simplicity of networks shares along with the superior economics of tape. Notre Dame’s IT department was able to re-purpose their existing tape library as a scale-out NAS using the Dternity appliance.

The athletics department and the university archives could all be given their own CIFS shares that plugged right into their workflows. The easy scalability quickly meets the ever-growing demand for capacity. Data recovery is automatically provided through replication to the Dternity Media Cloud. The Media Cloud is an offsite storage service that store two additional copies of Notre Dame’s content, giving them a total of 3 copies across two geographic locations.

As a starting configuration, Notre Dame built an archive designed to hold at least 750 TB using the Dternity NAS paired with a Spectra Logic LTO-6 library with replication to the Dternity Media Cloud service for added offsite protection.

The Benefits

To date, the IT team has seen an overall decrease in administrative time associated with backing up and archiving research data. Dternity simplifies data protection and disaster recovery by managing multiple file copies without requiring an additional backup application. There has been immediate cost savings already, and as the amount of data in the Dternity grows, the cost savings grows with it. It is significantly cheaper to keep archive data on tape as opposed to disk. Not to mention the security benefits now that less of the information that is supposed to be in the archive is being stored by users on flash drives and other user solutions. “Capacity and scalability were obviously very important to us, but Dternity provided so much more”, said Anderson. “Our customers are happy that their data is automatically copied offsite, and I never have to worry because our archive is fully protected for decades to come”. “The best thing though is how easily automated Dternity is and that our team for the most part never has to touch tape”.

MEET THE FASTEST LTFS NAS