Parallel Computing Systems plays an important role in the field of computer science, and are used for a wide range of computationally intensive tasks in various fields, including for example:
QStar HPC archive solutions can help organizations manage the escalating data capacities and complexities created by data-intensive applications such as simulation and modeling, hosted on the most popular HPC file systems.
QStar software solutions let you maximize productivity, improve storage efficiencies, and increase ROI.
HPC environments consume enormous amounts of high-performance data storage to process information for research needs. The current HPC systems operate on a multi-petabyte scale and that presents huge challenges and cost overheads. The vast majority of data is captured, used and re-used multiple times without modification, only the results vary based on the algorithms that change over time.
QStar Network Migrator identifies and archives data from HPC file systems. Selection of the files to be processed is driven by flexible policies based on standard file attributes such as modification and access times, file size, content, ownership, names, and extensions. Obviously the first time the system is installed it takes a while to build a base copy of the file system, but in subsequent periods only files that satisfy the policy are processed. Archiving performance in many respects depends on the type of HPC file system and the number of parallel processes being employed.
Archive storage is significantly less expensive than HPC storage but must offer enormous capacity and, in most cases, provide for replicated local and remote copies of the data for disaster recovery purposes.
QStar Network Migrator copies, moves, or migrates data to QStar Archive Manager or Archive Replicator. Archive Manager / Replicator is a software gateway solution allowing SMB, NFS or S3 protocols to store and retrieve data from archive storage that natively has no file system (such as tape libraries or Cloud).
Tape based systems solve this problem by providing vast capacity and low power consumption for data that is not actively used. The tape library is the least expensive and most reliable way to archive data. One of the other important benefits is that tape media can be easily copied for reliability and offsite storage.
On the downside, it may take several minutes to load the appropriate media to the drive and locate the right position to read the first file byte. Other storage targets for HPC data archiving can be lower-cost NAS disk systems but they require high power resources and monitoring of disk status.
Cloud systems may provide greater savings for local IT infrastructure management, but to achieve reasonable performance will probably mean using a private cloud which in the end will require infrastructure maintenance.
Many HPC organizations choose a combination of the above to gain performance benefits of private cloud (object storage) with cost-effective, long term tape libraries. Data is written to both technologies but only retrieved from the fastest. Over time – data can be purged from object storage if no longer regularly used and retained only on tape.
With QStar’s HPC storage solutions, organizations can drastically reduce investments in storage infrastructure to advance their HPC projects.
We can help you reduce cost, decrease vendor lock-in, and increase productivity of storage staff while ensuring accessibility and longevity of data.
Contact Us >