Blog

By Tony Ling, Director of Sales, Fujifilm Recording Media U.S.A., Inc.

After the initial shock and disruption of the pandemic, one industry that has rebounded nicely is the world of Media & Entertainment (M&E). Hollywood, video production, and post-production companies have adapted to making films in a COVID environment. At the same time, streaming grew significantly with most of the population homebound. Services such as Netflix, Hulu, Disney+, and Paramount all reported record subscriber growth over the last 15 months…..driving up the demand for new and original content.

Today, the retention and accessibility of digital assets and video content are incredibly vital to maintaining a competitive advantage. As a result, many modern M&E companies continue to assign starring roles to LTO data tape in their workflows to combat the rising expense associated with retaining and protecting capacity-intensive high-res content. 4K, 8K, 3D, and special effects can result in petabytes of storage for a single production!

With its high capacity, reliability, interchangeability, and security, the industry standard for deliverables has long been LTO tape…..this could be anything from daily camera footage, to post/edited work, approval copies, second copies, versions, final product, archival copies, etc. LTO tape is truly a defacto standard and an accepted part of the workflow in the M&E world.

Why are leading M&E companies turning to tape?

More M&E companies are recognizing the advantages of LTO tape, which can store massive amounts of data and combat ever-increasing storage costs across production, post-production, distribution, or archiving. Tape’s starring roles include:

• Extremely cost-effective with the lowest TCO in the industry
• Highly reliable with best in class bit error rates
• Secure with drive encryption and ease of offline storage to prevent cybercrime
• Portable for file sharing between locations
• Scalable to extremely large capacities with LTO-8 cartridge capacity now at 12.0 TB native and LTO-9 coming soon at 18.0 TB
• Open LTFS standard to allow for easy interchange of files
• Eco-friendly consuming 87% less energy than equal amounts of HDD

LTO tape is an ideal solution for M&E companies. LTO is an open format designed for interoperability and together with LTFS, provides easy data access and management—perfect for easy file share, high performance, and improved workflow.

So, the next time you are streaming Star Trek Discovery on Paramount+ or The Mandalorian on Disney+, just remember that somewhere along the way of the making of that show, an LTO tape played a starring role!

Recently my neighborhood had a rash of car break-ins by what turned out to be just a band of mischievous teenagers. But what struck me about this occurrence was the flood of homeowner video surveillance clips that appeared on social media and that were sent to the local police department to help identify the wrongdoers. It seems like everyone in the neighborhood has a home video surveillance system, perhaps to catch a doorstep package thief, or if nothing else, to catch the guilty dog walkers!

A Booming Market for Video Surveillance Solutions

Indeed, the video surveillance market is booming, not just in the relatively nascent consumer market, but in the commercial market and has been for a long time – in a much bigger way. The reasons for this include more affordable cameras with better resolutions soaring from 720p up to 4k and even 8k. In the meantime, video surveillance systems are finding more and more applications. Retail shopping malls, banks, hotels, city streets, transportation and highways, manufacturing and distribution operations, airport security, college dorm and campus security, corporate security, police body and dash cams, to name just a few – all need good quality video surveillance.

Video Retention Costs Soar

However, these higher resolution cameras have sent the costs of video retention soaring. So much high-resolution raw footage quickly fills up available hard disk drives commonly used to store or retain video surveillance content. According to a Seagate video surveillance calculator, an installation of 100 cameras recording eight hours a day at 30 frames per second, 1080p resolution, with a retention period of 90 days would require 2,006 terabytes of storage. That’s 2.0 petabytes of expensive, energy-intensive hardware. Those with unlimited budgets can simply add more disks. But everyone else faces tough choices: shorten retention periods? lower video resolution? reduce the number of cameras or frames per second? None of these support the goals of why the video surveillance system was installed in the first place.

Read more

The past 10 years have been marked by explosive data growth and demand for storage. Meanwhile, the tape industry has experienced a renaissance thanks to significant advancements in capacity, reliability, performance, and functionality that have led to new applications and key industry adoption. Here’s a look at some of the key milestones.

Capacity

• In terms of capacity, the decade started for LTO with LTO-5 at 1.5 TB native capacity and culminated most recently with LTO-8 at 12.0 TB and LTO-9 soon to be delivered at 18.0 TB.
• Enterprise tape formats started the decade at 1.0 TB native and are currently at 20.0 TB native.
• Barium Ferrite magnetic particles became a key enabler for multi-terabyte tapes and were demonstrated by IBM and Fujifilm in 2015 to have the potential to achieve 220 TB on a single tape cartridge. This signaled that tape technology had no fundamental areal density limitations for the foreseeable future.
• By the end of the decade, IBM and Fujifilm demonstrated the ability to achieve a record areal density of 317 GBPSI using the next generation of magnetic particles, Strontium Ferrite, with a potential cartridge capacity of 580 TB.

Reliability and Performance

• During the decade, tape achieved the highest reliability rating as measured by Bit Error Rate at 1 x 10¹⁹, even better than enterprise HDD at 1 x 10¹⁶.
• Data transfer rates for tape also improved from 140 MB/sec. in 2010 to an impressive 400 MB/sec.
• LTFS provided an open tape file system with media partitions for faster “disk-like” access and ease of interchangeability, making LTO a de facto standard in the Media & Entertainment industry.

New Applications and Key Industry Adoption

• Storing objects on tape became a reality with object archive software solutions offering S3 compatibility, objects can now move to and from tape libraries in their native object format.
• The concept of active archiving grew in popularity with tape as a key component complementing flash, HDD and cloud for cost-effectively maintaining online archives.
• Tape was recognized for its ease of removability and portability, providing air gap protection in the escalating war against cybercrime.
• Major U.S. hyperscalers began to rely on tape during the decade for both back-up and deep archive applications. In one well-publicized example, Google restored a February 2011 Gmail outage from its tape backups. Microsoft adopted tape for Azure later in the decade. Tape became firmly established as a competitive advantage for these and other hyper scalers based on its scalability, long archival life, lowest TCO, low energy consumption, and air gap security.

With this steady technological advancement over the last decade, tape has been recognized for its complementary value to flash, HDD and cloud in tiered storage strategies for managing data in the zettabyte age.

July 22, 2020

By Kevin Benitez

How much do you really know about your data? Is your data on the right storage type? How active is your data, and how is it being used?

From life sciences and media and entertainment to HPC/Research, higher education, government and consumer products, virtually ALL enterprises struggle to manage data with fewer resources and at less cost. Heterogeneous storage environments have added complexities, costs, and made it difficult for IT managers to manage data.

Don’t let multi-vendor storage silos get in the way of effective data management.

This webinar series goes beyond just organizing your data. Throughout three short webinars, you’ll learn about how to take control, protect, and manage your data – all while enhancing workflow and reducing costs.

Join Floyd Christofferson, CEO of StrongBox Data Solutions, in a webinar series that will teach you how you can make the most of your data:

1. Take Back Control of Your Data + LTFS

Don’t let multi-vendor storage silos get in the way of effective data management.

July 28, 2020 12:00 PM – 12:45 PM Eastern Time

2. Reduce Costs & Increase Data Protection!

How to Better Manage Data Growth in a Multi-Vendor Storage Environment.

August 4, 2020 12:00 PM – 12:45 PM Eastern Time

3. Workflow Magic!

Techniques to better use your data and not waste time trying to wrangle it.

August 11, 2020 12:00 PM – 12:45 PM Eastern Time

Register Now

By Rich Gadomski

I just spent a full day at a meeting of the Active Archive Alliance and as I was flying home it occurred to me that it’s time for data storage managers to rise up from the sleepy status quo of buying more disk arrays to address runaway data growth problems. It’s time to wake up and smell the sweet aroma of freshly made modern data tape (sort of like that new car smell if you don’t know).

Why do that you ask? Because best practices and undeniable facts say so. Consider the following:

Data goes through a lifecycle from hot to cold, that is to say from a period of active use to a period of inactivity. This can happen in as little as 30 days or less.

Inactive data should not stay on primary storage devices. It takes up space on expensive storage media, consumes more energy and adds to the backup burden.

What to do? Delete it? You probably can’t get permission to delete it, all data is now potentially valuable with new artificial intelligence (AI) and analytic tools emerging to derive value from that data. But you can move it and stop copying it!

Where do you move it to? Put it in an active archive consisting of low cost disk cache and even lower cost long term storage like a high density automated tape library. To store one petabyte of data for 10 years in a tape library will cost around $220,000 depending on your TCO variables. Alternatively, you could spend $900,000 on HDD and around $1,300,000 for cloud. Need more capacity? Tape libraries easily scale by adding more slots and tapes. You can export full tapes and plug new ones in. Move the full tapes offsite and get the benefit of air gap since the data is physically isolated from other networks. At least you know that data can’t be accessed and held for ransom.

Getting end user access requests for that data all of a sudden? Move it back to disk cache and serve it from there. When done, move it back to the tape library. Tape is super-fast, 360 MB a second and file access is made easier and faster with LTFS.

How to orchestrate all this? Intelligent data management solutions help move data automatically. Leverage metadata and AI tools to analyze files and move them off primary storage if they don’t belong there.

Does this sound like a tiered storage strategy? It is and it’s also known as an active archive. This is a best practice used by the biggest and most advanced data generating companies in the industry. If it works for them, it will work for you too.

There’s a lot of hype in the storage industry with lots of folks looking for new, better ways to do things. But some things are tried and true, like tape, with the benefits of constantly evolving capacities, performance, reliability and long term archivability. So wake up and smell the tape…put your data where it belongs and get on with your day!

By Andy Feather

I often hear from customers that are sitting on scores of legacy tapes with unknown contents beyond a generic “business data” level, and 99+ percent of them are not known at a granular level. As we all know too well, disaster recovery backups morphed into unintentional data archiving these past 10 – 15 years thanks to litigation and government regulatory investigations, along with general business obligations to retain certain records.  The duty to preserve has forced businesses to preserve backup tapes if at least one file on the tape might be under some form of preservation obligation.  The IT staff almost never has the equipment or human resources to perform targeted restores of data under preservation and stack it together with other similar data, so they take the easy way out: buy more tape and retain existing tapes vs. overwriting their contents.  Companies change backup software providers and migrate to newer backup platforms and get stuck paying maintenance and support for software and hardware they no longer use, but might one day.

An additional problem lies in the fact that companies are waking up and realizing that while tape as a storage mechanism is a great value, the real estate and costs associated with parking and retaining them in mass quantities can add up.  In response, companies like Seagate and TapeArk offer to move large volumes of data into the cloud, but does this provide value to the customer?  Why pay to migrate thousands of tapes to the cloud on the chance that you might one day need to access them?

So I came across a neat solution to this problem from a service provider/software developer named SullivanStricklerout of Atlanta. They recognize the gap between the status quo and the cloud and created TRACS/TDF and TRACS/TSF.  TRACS stands for Tape Restoration and Cataloging System, TDF for Tape Duplicate File and TSF for Tape Session File. TDF and TSF files are both file containers which consist of data from legacy backup tapes, regardless of the source tape type and backup software format. TDF and TSF provide customers with a catalog of the contents of the tape and the ability to immediately restore the contents of the once backup tape, now TDF or TSF file, and/or stack and store the TDF/TSF files onto newer, higher capacity media using LTFS or some other backup software.

The economics of tape stacking have been explored for years, but the “value” of the exercise provided little ROI until 6.0 TB LTO-7 tapes arrived.  The combination of reducing the storage costs associated with 60 LTO-1 (100 GB) tapes and replacing them with one LTO-7 tape, along with the increased value of discovering the contents of long forgotten backups and never having to pay licensing and support fees for technologies you no longer use, combine to provide the justification for businesses to begin to explore a stacking/migration effort.

Some customers ask, “But if I am going to undertake this effort, why do I need to migrate everything instead of only what I need to keep?”  This is a very valid question, and is a good segue into the differences between TRACS/TDF and TRACS/TSF files.

TDF or Tape Duplicate File, is a byte-for-byte copy of the source tape, with the addition of a catalog of the tape contents appended to the file.  Files ranging in quantity from one to all can be restored from a TDF file, and as a bonus the conversion process is reversible.  This means that customers who convert from tape to TDF format can ultimately rewrite the data back out to tape so that it can once again be used by the backup software which originally created the tape, should there ever be a need.

TSF, or Tape Session File, differs slightly from a TDF file.  Whereas a TDF file is a duplicate copy of an entire tape in one logical volume container, a TSF file is an individual logical session container from a tape.  A TSF file can be created for one backup session, up to all of the backup sessions on the tape.  TSF files are exciting because of the business value they provide.  TDF files provide great value due to the stacking and cataloging elements, but TSF files allow users to pick and choose which backup sessions to retain and which can be deleted.  If a company’s preservation requirements are such that they need to retain all backups of their email system and their file servers, but not their domain controllers, print servers, departmental databases, etc., then TSF files allow them to do this by breaking up the “if I need one file I need to keep the entire tape” limitation.   This process results in an even larger business value than TDF through the reduction in risk associated with retaining data which need not be retained, and since not all sessions will be retained by customers, the reduction in data volume is multiplied.

Additionally, with one eye on the growing number of state, national and international regulations concerning data privacy and information governance, such as the EU’s General Data Protection Regulation (GDPR) or California’s Consumer Privacy Act, TSF allows for the defensible deletion of files stored within backups, without impacting the remaining backed up files.  This type of targeted deletion of data originating from tape is quite unique, and all performed without restoring the data from a single tape.

Of course there are other solutions but I like the simplicity and logic of TRACS/TDF and TRACS/TSF. Certainly it’s more practical and affordable than what Seagate and TapeArk propose!

Storytelling is a central facet of society that may have changed formats over the years but will never become obsolete. In today’s digital world, broadcasters and television networks focus on creating relatable stories to connect with their audiences, and they can’t do that without a wealth of readily available content.

MLB Network is the source for baseball stories of all kinds, from live games to studio shows and feature programming. Launched on January 1st, 2009, MLB Network is growing fast, reaching more than 70 million households today, delivering the best of America’s national pastime, all the time.

The Challenge: Digital Content Storage and Management

“MLB Network’s goal is to bring baseball to our audience every night with the highest levels of production quality, focus and enthusiasm throughout the year,” said Tab Butler, Director of Post Production and Media Management at MLB Network. “To accomplish this, we need constant access to our archives and current live game content. We need all information from every game securely stored and easily accessible.”

With multiple recordings of every game, along with multiple audio sources, and pregame, post-game and isolated camera feeds, it is not uncommon for MLB Network to record more than 3,000 hours of content per week. That content is then categorized and cataloged for future use, using the Emmy Award-nominated media asset management DIAMOND System. When the baseball season comes to a close, MLB Network continues to deliver baseball news 24/7, with special programming about a team, player or other happenings in the sport. These individual projects require systematic archival that supports precise selection and instant access of specific files. The challenge is how to empower diverse departments to directly access their projects, without heavy IT support.

MLB Network Deployed StrongBox for Project-Based Workflow

As an early adopter, MLB Network deployed a custom-developed StrongBox to manage archived projects in late 2011. StrongBox is a vendor-neutral, fully portable data vault for long-term file retention. Functioning as standard network-attached storage (NAS), StrongBox employs Linear Tape File System (LTFS) media as the principal storage medium to save money and empower a file-system view of all archived content. An internal disk cache enables rapid file access. With drag-and-drop functionality, StrongBox makes accessing archived projects easy for MLB Network, delivering content on-demand to multiple, simultaneous users. Since MLB network is a 24/7 operation, the production staff uses a SAN-based Final Cut Pro (FCP) platform to develop programming that is updated throughout the season with the latest information. When these projects are ready for archival, the video, audio and revision files, along with their metadata, are stored in StrongBox. Having the ability to recall an archived show, and repackage it with current information, utilizing tape as the storage medium, reduces the storage costs for the archived content. “StrongBox has been natural for streamlining this type of project-based storage. Instead of keeping projects on spinning disk, we’re able to offload to StrongBox,” explained Butler. One of Butler’s key initiatives is finding ways to better automate the media management environment, allowing different departments to manage their own archival data instead of relying on his Media Management team to store and retrieve files. With StrongBox, editors have direct access to archived projects in real-time, without having to depend on a Media Management operator to retrieve files.

High-Capacity, Low-Cost LTFS Tape for Proxy

Butler said that for the 2009 baseball season, 25 terabytes of spinning disk storage was required for the video proxy data, and for the 2010 baseball season, 32 terabytes was required. Although this proxy information requires long-term storage, it is accessed very infrequently. Thus, the operational costs for keeping this much data on spinning disk become extremely expensive. Even though it is long-tail content, it cannot be taken offline.

Through the custom-built, DIAMOND System, MLB Network logs and categorizes HD recording by viewing the video using proxy video files, which are recorded in real time. The bulk of recorded HD Video content is stored within an LTO library, and is searched and accessed using DIAMOND and the Grass Valley Aurora systems. Thus, Butler is investigating ways to use tape to further drive down the costs of his long-tail proxy content which is currently on spinning disk.

“If I look down my future roadmap, my proxy environment is going to continue to grow year over year for the lifetime of the archive,” explained Butler. “Getting the proxy on LTO-5 media is much more cost-efficient for long-tail content.” With the introduction of LTFS, tape can be partitioned and indexed on a file level. This brings significant opportunity for media and broadcast companies. For MLB Network, LTFS brings the capability to efficiently maintain an accessible archive at the file level, while eliminating the heat generation, cooling requirements, spinning drives and other operational costs associated with disk storage.

“StrongBox is a very unique product, with flexibility that makes it functional in multiple use-cases for MLB Network,” Butler continued. Currently, the MLB Network video archive consists of 10 to 12 petabytes of stored HD content, and 275 terabytes of proxy content. A migration of that much data would be a significant undertaking. With the ability to scale up to 35 petabytes, StrongBox delivers low-cost, high-capacity storage with high-performance access that could provide a cornerstone in the foundation for MLB Network’s biggest business asset – its programming.

The Bottom Line

MLB Network delivers exciting, engaging baseball stories 24/7, 365 days a year. With a massive content archive that will only continue to grow, Tab Butler knows that LTFS tape is a cost-efficient and scalable way to manage MLB Network’s digital records. While editors constantly juggle multiple projects with demanding deadlines, StrongBox facilitates a project-based workflow, integrating with the editing environment to allow end users more direct access to their archived content. Ultimately, StrongBox helps MLB Network spend more time creating and delivering award-winning baseball stories and less time worrying about how to manage data.

MEET THE FASTEST LTFS NAS

With the arrival of Active Archive, much higher data rates, LTFS, RAIT, and RAO capabilities now in place, the tape industry is making significant strides in delivering much faster initial access times and throughput levels. Read more about how tape is accelerating performance here.