FUJIFILM INSIGHTS BLOG

Video surveillance has become an essential tool in many industries, from law enforcement to retail, transportation, and critical infrastructure. The ability to capture and analyze video footage has enabled organizations to enhance their security, prevent crimes, optimize operations, and make data-driven decisions.

However, with the proliferation of high-definition cameras, the exponential growth of video data, and the increasing demand for longer retention periods, managing video surveillance storage has become a significant challenge for many IT and security departments.

Rather than storing all video data on expensive, energy-intensive, high-performance storage devices such as hard disk drives, organizations can leverage a 2-tiered approach that provides for the quick access of the most recent video and the availability of all recorded video no matter when it was originally stored. This approach can significantly reduce storage costs, optimize system performance, reduce carbon footprint, and simplify video data management.

Read more

Recently I had the opportunity to meet with a newly appointed Chief Sustainability Officer (CSO) at a major scientific research organization. In this new position that seems to be trending across many industries, this CSO has been tasked with the overall responsibility for understanding the organization’s energy profile and how its carbon footprint is spread across its many different departments and then to figure out what actions can be taken to help achieve its carbon reduction goals. With this understanding in place, the CSO will proceed to put downward pressure on the department heads to make meaningful change.

Given this background, the purpose of my call was to explain the role that today’s modern data tape can play in reducing power consumption and associated carbon footprint in data centers. Since this CSO really had no experience in large scale data operations, she was eager to listen given her need to start looking at every department to quickly identify opportunities for energy and carbon reduction. She will go after the low-hanging fruit first, but eventually no stone will be left unturned if her organization is to meet their aggressive sustainability goals.

Read more

The Active Archive Alliance recently launched its 2023 State of the Industry Report: Effective Data Management Through Active Archives. The report focuses on how an active archive model solves data growth challenges and provides today’s organizations with a way to manage and access their archival data cost-effectively.

At the center of an active archive resides an intelligent data management system. This software system plays the central role of automatically placing data where it belongs for cost, performance, and workload priorities. The data management layer uses technologies such as metadata and global namespaces to make data accessible, searchable, and retrievable on whatever storage platform or media it may reside.

Key benefits include:

• An essential active archive principle asserts that today’s enterprises need online access to legacy data. Some use cases may need fast access; in other instances, a longer retrieval time is acceptable. The organization determines its access needs and permissions for users and groups.
• Cost Savings. Through intelligent data management software, an active archive moves inactive data to low-cost storage. For some organizations, data management software can tier older data to warm storage such as hard drives. Then when, by policy, data has aged sufficiently, files can move to more cost-effective storage such as economy disk, tape, optical or even the cloud.
• An active archive can supply a wide range of security features and cyber resiliency capabilities to secure and protect data from cyber threats facing today’s businesses and institutions. Because archival data typically remains unchanged, administrators may use WORM (Write Once, Read Many) or view-only mode features to prevent data from being deleted or overwritten, safeguarding data’s integrity, availability, and confidentiality.

In addition to these benefits, an active archive provides technical leaders the flexibility to adapt to new industry trends and growth areas such as sustainability, artificial intelligence, and edge computing. This flexibility helps enterprises thrive with possibilities for market leadership, increased revenue, and competitive advantages.

To download the 2023 Active Archive Alliance Annual Report, click here.

The National Association of Broadcasters (NAB) held their annual conference this week in Las Vegas. Some 68,000 attendees from broadcasters to screenwriters, advertisers and streamers, to producers and filmmakers were eager to learn about the latest in media and entertainment technology. Some 1,200 exhibitors set up shop in the Las Vegas convention center, including FUJIFILM with our FUJINON lenses, and Recording Media in the form of LTO tape technology.

Read more

  I attended the International Security Conference West (ISC West) in Las Vegas last week. The show was well attended with some 20,000 visitors packing the aisles and mobbing the booths of some 650 vendors. The show covers everything from video surveillance to computer and communications security, to physical plant security, loss prevention, and more.  

A few items struck me as a first-time visitor to the show, namely, entire booths dedicated to gunshot detection devices and others dedicated to things like drone detection. It’s a sad sign of the times when security professionals have to install gunshot detection devices to improve response time and mitigate deadly intrusions. In the case of drone detection, if you are being spied upon from above, no worries drone detection radar can now electronically jam unmanned aerial vehicles and can even deliver a kill shot as a last resort.   

The other item most relevant to me was the fact that this industry stores almost all of its video surveillance content on expensive, energy-intensive hard disk drives. That’s a problem in general, but even more so when security professionals are being besieged by any number of new budget-draining threats like gun-toting intruders and nefarious aerial snoopers.  

Read more

Reflecting on World Backup Day coming up on March 31st, I recall coming across the following quote that recently appeared in an industry newsletter from an IT executive that said:

“For years IT administrators have worked hard to back up an increasing tsunami of data, and with each passing year, that has become harder to manage. In some cases, backup has been abandoned altogether. That is a precarious place to be.”

Indeed, a very precarious and dangerous place to be considering the increasing value of data for data-driven organizations. Today’s modern and highly advanced data tape systems can help solve the problem.

Here are five of the top reasons why tape should be part of the backup process:

Read more

John Monroe, a long-time storage industry expert and Gartner analyst, now an independent consultant  with his own company (Furthur Market Research), recently published a new report entitled “Preservation or Deletion: Archiving and Accessing the Dataverse”. This report is a follow-up to John’s initial report entitled “The Escalating Challenge of Preserving Enterprise Data”, and is co-sponsored by Fujifilm, IBM and Twist Bioscience. This new report looks at likely growth rates of new enterprise capacity shipments required to store the ever-expanding “dataverse” and manage the swelling installed base of enterprise-grade SSD, HDD and tape media from 2023 to 2030. The findings and conclusions in John’s report clearly suggest that the status quo in storage strategies is not sustainable. Below are some summaries and excerpts taken from the report and a link is provided to view/download the full report.

Relentless Growth of the Dataverse

John provides a forecast for SSDs, HDDs and tape capacity shipments and the growing installed base from 2023 to 2030. With a CAGR of 30.7%, new shipments of enterprise storage capacity will hit 1.74 ZB in 2023 (that’s up from .95 ZB in 2020) and exceed 11.0 ZB in 2030. Meanwhile, the active installed base of enterprise storage will grow from 6.4 ZB in 2023 to 35.7 ZB in 2030. In a worst-case 25% CAGR scenario, new shipments of enterprise storage capacity will grow to 8.0 ZB while the active installed base expands to 26 ZB in 2030.

However, those forecasts could change dramatically if a not unlikely growth rate of 35% or even 45% should unfold. At 35% CAGR, we would see new capacity shipments of 14.7 ZB with an active installed base of 45 ZB in 2030. (Note: it takes 50 million 20 TB HDDs or 22 million LTO-9 tapes at 45 TB compressed capacity to store just one single zettabyte).

Evolving Data Temperatures

John also provides a breakdown of data temperatures depicted in a classic pyramid with Hot data at the top, followed down the pyramid by Warm, Cool, Cold and finally Frozen data layers. By 2030, the Cold and Frozen data layer will be the largest segment at 61% of stored data. This is largely  because of the answer to the implied question posed in the title of the report “Will we preserve or delete our data?” In John’s surveys of end users across different vertical markets, almost all of the IT managers he spoke with specified “indefinite” retention periods for the vast majority of their data, even if frequency of access declined to seldom if ever. We will be storing and maintaining an ever-increasing amount of enterprise data that has aged for more than five years.

Massive Revenue Opportunity Ahead

With a majority of data being stored long term in Cold and Frozen layers requiring lower cost per GB and more energy efficient technologies, John conservatively estimates revenue for enterprise storage devices in these tiers will range from $8.8 B to $15.7 B in 2030, up from $5.1 B in 2023. This bodes well for new generations of tape and emerging technologies like DNA storage that will change the current trend of storing so much of this type of data on expensive and energy intensive SSDs and HDDs.

On Sustainability

The report goes on to show that energy consumed by maintaining the installed base of SSDs and HDDs between 2020 and 2025 would consume over 15,000 megawatts of power while the tape installed base for the same period would consume just 18 megawatts, an 838 X difference. In John’s own words:

“It is obvious that HDDs and perhaps a significant number of SSDs are handling far too much of the Cold/Frozen workloads at far too great a cost/GB while consuming an inordinate share of available energy”.

Limited HDD and SSD Production Capabilities

Because the HDD makers have fiscal concerns about investing unprofitably in future CAPEX in the face of uncertain demand and growing SSD incursions, John fears the HDD industry will not adequately invest to be able to deliver ~5 ZB, much less ~8 ZB, of enterprise-grade media per year from 2028 to 2030. And given the recent precipitous price erosions—the price for raw NAND dropped by more than 70% during 2H22—and the inevitability of future supply/demand imbalances and the attendant price fluctuations, John also has growing doubts that the NAND industry will spend the necessary hundreds of billions of dollars to be able to deliver ~1 ZB, much less ~2-3 ZB, of enterprise-grade SSD storage capacity per year from 2028 to 2030. But even new shipments of ~6-10 ZB of expensive, enterprise-grade SSD and HDD media may be insufficient to meet global demand in 2030.

Resurgence in Tape Shipments

The report goes on to say that with limited SSD and HDD production capabilities looming and the increasing need for cost-effective and sustainable storage, the demand trend for new generations of tape, DNA data storage and even optical technologies may be altered drastically. Regarding tape specifically and considering recent hyperscale market adoption, the report suggests:

“There will be a resurgence in tape shipments for a variety of reasons based on expanding demand on multiple fronts, relative data temperature and time-to-data needs based on access frequency, and lower costs of data retention and power consumption, as well as limited HDD and SSD production capabilities. Tape could well grow to at least two zettabytes delivered by 2030”.

In Conclusion

The data centers of the future will need everything the SSD, HDD and tape industries can manufacture and deliver, as well as requiring new DNA and perhaps other enterprise storage technologies. Availability and sustainability challenges, combined with the costs of managing the dataverse over increasingly lengthy time periods, will create new use cases for existing storage technologies and demand the creation of new, more cost-effective, and power-efficient storage technologies.

To read the full report:

https://asset.fujifilm.com/www/us/files/2023-03/772c53c57d3e6022681fb5231e7df79a/FMR_Preservation_or_Deletion_WP_March_13_2023.pdf

Read more

The origin of the phrase “any port in a storm” is not very clear or well documented, but the definition is.  When you are facing some difficulty, you need to accept any solution without waiting for a perfect solution.

Such may have been the case at the outset of the great COVID pandemic of 2019 (and 2020, 2021, 2022, and 2023 too). When it came to dealing with the challenges of storing ever-increasing volumes of valuable data and the new phenomena of work-from-home, the cloud looked like a pretty fast and safe port in the COVID storm, and in some cases, it was and still is.

Prior to COVID-19, the term “cloud repatriation” appeared often in the press as it turned out that cloud was not a panacea for everything. But COVID understandably created short-term tactical storage strategies often resulting in a flight to the cloud.

However, the notion of cloud repatriation seems to be making a comeback, perhaps as a result of the prevailing conditions of economic uncertainty, geo-political tension, inflation, tightening budgets, cyber threats, and other factors. But it’s no longer a question of cloud vs on-premises. Savvy IT managers have the option of a more strategic hybrid cloud approach where the best of public cloud plus on-premises infrastructure provides maximum flexibility and value.

Read more

A new white paper written by Dr. Shawn Brume of IBM entitled Reducing Risk and CO2e in the Disposition of IBM Physical Tape Media does a great job in answering what to do with tape media at end-of-life (EOL).

The report is timely as the industry is gaining interest and awareness of energy consumption in data storage driven by ESG directives. For example, a recent white paper by Brad Johns Consulting, Improving Information Technology Sustainability with Modern Tape Storage demonstrates the sustainability and cost-benefit of tape:

Read more