Whitehead Cracks the Code on Cost-Effective Storage

Whitehead Institute is a world-renowned non-profit research institution dedicated to improving human health through basic biomedical research. By cultivating a deeply collaborative culture and enabling the pursuit of bold, creative inquiry, Whitehead fosters paradigm-shifting scientific achievement. For more than 30 years, Whitehead faculty have delivered breakthroughs that have transformed our understanding of biology and accelerated development of therapies for such diseases as Alzheimer’s, Parkinson’s, diabetes, and certain cancers.

The Challenge

The Whitehead Institute, based in Cambridge, Mass., takes on some of the most complex and important medical and scientific challenges ever presented to mankind. In the 33 years since its founding, it has become one of the world’s leading molecular biology and genetics research institutes, employing multiple National Medal of Science winners. In fact, the Whitehead Institute was a key contributor to the 13-year Human Genome Project, a groundbreaking study that unlocked an entirely new understanding of how humans react to viruses, bacteria and drug therapy.

Research at the Whitehead Institute generates an enormous amount of data. Genomic sequences and microscopy images alone can add up to multiple terabytes a week. Information is further extracted from the raw data using a computing cluster that leads to the creation of processed data files. This all translates into a unique set of challenges for the Institute’s IT team. Like the scientists they support, the IT team has had to address their challenges with innovative and experimental approaches.

“The scientists do everything from basic cellular process research to cancer and other diseases research,” said Paul McCabe, Senior Unix Systems Administrator and Data Center Specialist. “It varies widely, but the common denominator is that our research generates a huge amount of very valuable data.”

Due to the historical implications of their research, scientists at the Whitehead Institute constantly have to look back at previously collected data to forge ahead with their work.

“We tend to process data pretty heavily, and we have long-term data retention requirements,” said McCabe. “We not only store the data while it’s being actively processed by our researchers, but we also need to archive that data long after research papers are published in case the data behind the papers are ever challenged.”

As the Institute’s operations have become more dynamic and strenuous in nature, the legacy systems in place have had trouble keeping up with the increased workload and demand.

“Our organization had become a 24-hour endeavor, which was a challenge that was becoming more and more difficult to manage,” explained McCabe. “We were backing up for eight hours a day, duplicating for eight hours a day, and archiving the remaining eight hours. The equipment was being pushed to its limits, and if anything went wrong… we were simply out of hours.”

The Solution

As a result, McCabe and the IT team began researching high capacity data archiving alternatives that could meet their scalability, reliability and simplicity needs. At an IT tradeshow, the team was introduced to the Fujifilm Dternity, a data archiving system that combines the simplicity of disk and the economics of tape into a highly scalable, easy-to-manage solution.

“We also liked the way Fujifilm structures its licensing model in large bands, rather than the ‘by the terabyte’ model offered by other vendors. Overall, it matched very well with our requirements.”

Currently, the Whitehead Institute IT team is storing 171 TB of unique data on the Dternity NAS, with room to grow to more than 400 TB.

The Benefits

To date, the IT team has seen an overall decrease in administrative time associated with backing up and archiving research data due to the system’s ease of use and scalability. There has been some cost savings already, but as the amount of data in the Dternity grows, the cost savings grows with it. It is significantly cheaper to keep archive data on tape as opposed to disk. “Capacity and scalability were obviously very important to us, but Dternity provided so much more,” said McCabe. “Our backup team is thrilled with how easy the system is to manage and how it frees them up to focus on other tasks, but I would say the most noticeable benefit is the overall peace-of-mind the Dternity provides us. We’re dealing with critical data, and I never have to worry because it is fully protected, backed up and available when needed.”

The University of Notre Dame, based in South Bend, Indiana is a world class university with students that represent all 50 states and over 100 countries. Rigorous academics, NCAA Division I athletics, and numerous events and activities create an environment in which students are ignited with a passion to learn and to live to their fullest capacity.

Consistently ranked in the Top 25 institutions of higher learning by US News & World Report, Notre Dame has over 10,000 students, 4 colleges, 67 undergraduate programs, more than 50 graduate programs, and is home to the famous Fighting Irish Football team that draws fans from all around the world.

The Challenge

“The mission of the Office of IT is to serve as the trusted IT partner to all the Faculty, Staff and Students” says Mike Anderson, Storage Engineer at the University of Notre Dame. “It can be very difficult to satisfy their increasing demands for storage”. Historically IT was responsible for providing enterprise class storage for students, faculty and staff. This was primarily for user accounts, database and email. Large file storage was discouraged and requests for such storage were often denied. “Sometimes we would get requests from a department to store 100 TB or more” says Mike Anderson. “We simply did not have a solution that could accommodate a large archive at that point”.

“We needed an affordable storage solution that could scale to beyond a petabyte and provide data protection to all archived content”
– Mike Anderson, Storage Engineer at the University of Notre Dame

By 2014 it was clear to IT that something had to change. Requests for large file storage was growing and so was the need to archive them. Primary sources of this big data were the Athletics Department that was using more and more cameras to film all their games. Another source was the ever growing University Archives that was making an official effort to digitize and preserve historical data for future use. “We needed an affordable storage solution that could scale to beyond a petabyte and provide data protection to all archived content” said Mike Anderson.

The Solution

“After evaluating many solutions we decided to go with the Dternity NAS and Media Cloud replication service”. The solution is built to fit into existing environments and deliver the simplicity of networks shares along with the superior economics of tape. Notre Dame’s IT department was able to re-purpose their existing tape library as a scale-out NAS using the Dternity appliance.

The athletics department and the university archives could all be given their own CIFS shares that plugged right into their workflows. The easy scalability quickly meets the ever-growing demand for capacity. Data recovery is automatically provided through replication to the Dternity Media Cloud. The Media Cloud is an offsite storage service that store two additional copies of Notre Dame’s content, giving them a total of 3 copies across two geographic locations.

As a starting configuration, Notre Dame built an archive designed to hold at least 750 TB using the Dternity NAS paired with a Spectra Logic LTO-6 library with replication to the Dternity Media Cloud service for added offsite protection.

The Benefits

To date, the IT team has seen an overall decrease in administrative time associated with backing up and archiving research data. Dternity simplifies data protection and disaster recovery by managing multiple file copies without requiring an additional backup application. There has been immediate cost savings already, and as the amount of data in the Dternity grows, the cost savings grows with it. It is significantly cheaper to keep archive data on tape as opposed to disk. Not to mention the security benefits now that less of the information that is supposed to be in the archive is being stored by users on flash drives and other user solutions. “Capacity and scalability were obviously very important to us, but Dternity provided so much more”, said Anderson. “Our customers are happy that their data is automatically copied offsite, and I never have to worry because our archive is fully protected for decades to come”. “The best thing though is how easily automated Dternity is and that our team for the most part never has to touch tape”.

MEET THE FASTEST LTFS NAS