Active Archiving with Qstar

The Future Of Archive Storage Is Tape, No Doubt About It!

For data archiving, tape has no competitors, unless you do not care about the cost savings, long-term dependability, interoperability, reliability 100,000 times that of hard drives (SATA) and much more. Today’s LTFS as NAS offers the most cost-effective long-term data archiving solution available on the market, with a TCO that can vary, depending on the capacity acquired, from 50 to 300% lower than comparable disk-based archiving systems.

These are the facts, as distinct from the hype about tape peddled by disk vendors. Whoever thinks that tape technology is over is grossly mistaken. To understand why this technology represents the best data archiving solution, we need to clear up the confusion between the meaning of the terms backup and archiving. Data backup and archiving serve two quite different purposes.

  • The purpose of a backup is essentially to enable you to restore the latest copy of the data to immediate availability as quickly as possible
  • The purpose of an archive is to store data for the medium or long term

If you need to store data safely, for several or many years, at a very competitive cost, especially as regards TCO, LTFS as NAS has a number of unique benefits over medium-long-term archiving on disk-based systems. This is especially apparent once you realize that more than 80% of data stored on primary disks is no longer or only rarely accessed again. As if this were not enough, this dormant data is also included in backups and migrated repeatedly for years to new and costly storage devices. This implies a huge waste of money and manpower; a worrying scenario, above all for those wrestling with IT budgets and shrinking staff levels.


Transferring Recorded Media Between Different Operating Systems

The increased popularity of Active Archive tape-based architecture was driven by the introduction of the Linear Tape File System (LTFS) standard initially introduced with LTO-5. The LTFS format complies with standard ISO/IEC 20919:2016. The format guarantees media interchange between operating systems and different archive system manufacturers who have adopted it.


Read-Write Interoperability Between Different Operating Systems

LTFS divides the tape into two separate areas known as partitions, one for metadata (file information such as name and size), and the other for storage of the data itself. When an LTFS tape is mounted, the partition containing the metadata is read and the catalog of contents becomes available to applications.

The QStar LTFS as NAS solution can import LTFS media created outside of the main archive, e.g. from a remote site or another manufacturer who has adopted this standard. LTFS media can be imported in order to be transferred between different remote sites. The data can be copied to the allocated LTFS as NAS Active Archive and then returned to the field for reuse or stored in the LTFS as NAS Active Archive as a separate element of the archive.

Likewise, LTFS media created within the environment can be exported and read by other solutions based on the same LTFS standard. The key advantage of the LTFS standard is the option of transferring recorded media between different operating systems, applications and platforms that employ this format so that the tape can be used just like a removable disk. You can even drag and drop files from folders on the server to the tape using a standard operating system directory.

LTFS Indexing – How does it work?

Data Sharing Between Different Operating Systems


Linear Tape Open – LTO

LTO tapes guarantee an archival life exceeding 30 years, support a million “passes” and 20,000 write cycles per tape.

Technology developments continue at a rapid rate in tape and specifically LTO technology. Improvements in the recording mechanisms, increased capacity, higher data integrity, plus mechanical durability and reliability. An LTO tape is 1/2 inch wide and passes longitudinally across the tape drive head without touching it, from the beginning of the tape to the end; recording until the tape is fully unwound. Following this the media is rewound, writing to an adjacent area on the tape, creating a kind of serpentine pattern. Attached to each read/write head are two sensors that allow very precise head positioning to the relevant data areas using service tracks on the LTO media.

In order to guarantee a high level of protection, the tape edges (most liable to suffer mechanical damage) are not used for recording. The read/write system utilizes multiple magneto-resistive heads that change polarity each time the tape is wound in a different direction. After writing, data is immediately checked by following read head for real-time detection of potential errors.


Compatibility Between Different Generations Of Products


When choosing an archive technology, besides specific technical features, it is also important to consider its market penetration to avoid being left with an out-of-date or, worse still, discontinued product in the near future. LTO technology not only guarantees a market approved standard, but also offers a wide choice of manufacturers for media, drives and tape libraries. No other current storage technology can provide such a guarantee for the future. With tape libraries, a large number of manufacturers offer architecture and capacity designed to meet all operational requirements.

Another important aspect of this technology is the guaranteed compatibility between different generations of products; the LTO standard provides.

LTO Technology Options – Rewritable or WORM

LTO rewritable media in general offers very high-level data archive security. For all cases where data must not be altered accidentally or must comply with regulatory archiving standards, there is also a non-rewritable LTO WORM version. The LTO WORM cartridge was developed to allow organizations to store critical data which once written cannot be overwritten or tampered with.

Another benefit of LTO WORM media is its compliance with various regulatory requirements including the 2002 Sarbanes-Oxley Act, the Health Insurance Portability and Accountability Act of 1996 (HIPAA), and SEC Rule 17-a-4(f). The data can be also encrypted for an even higher level of security. LTO tapes can be encrypted with 256-bit AES-GCM encryption, and the data can be compressed prior to encryption to maximize tape capacity and provide high performance archiving.

An LTO WORM Cartridge is identical to a normal LTO cartridge except that its LTO-CM chip identifies it as WORM to the drive and the servo track is slightly different to allow verification that the data has not been modified. It may also come with tamper-proof screws. WORM-capable drives immediately recognize WORM cartridges and include a unique WORM ID with every dataset written to the tape. There is nothing different about the actual tape medium in a WORM cartridge, though typically, WORM cartridges do have a different color packaging.


LTO Technology, A Paragon of Reliability

An unbeatable level of integrity of data archiving with Linear Tape Open (LTO) technology. Linear Tape Open (LTO) is a standard introduced in 1998 and jointly developed by IBM, Hewlett-Packard and Seagate with the aim of bringing the concept of open standards and multivendor compatibility to the tape market (as well as providing high reliability, scalability and performance levels). LTO tapes guarantee an archival life exceeding 30 years, support a million “passes” and 20,000 write cycles per tape and an MTBF (Mean Time Between Failure) of 250,000 hours at 100% duty cycle.

The various types of data integrity verification also include block-level checksum and CRC (cyclic redundancy check) performed directly by the LTO drive. LTO drives perform on-board data verification of each write operation to physical tape; the LTO drive head is engineered so that each block written to tape is reread in real-time by another section of the same head. This makes it certain that data written to the medium (tape) is correct. In addition to this check, each block undergoes a CRC in order to identify a unique numeric value for that particular file block. During reading operations, these CRC values are also used to detect and correct any errors. If using QStar TDO file system an additional checksum is carried out on fixed-size objects within the files. Checksums are calculated for each object and automatically checked upon each reading operation.


Exceptional Long-term Reliability and Integrity for Data Archived in Tape Libraries

Besides the media manufacturer’s guarantee as to the long-term (30 years) integrity of data recorded on the medium (tape), QStar recommends media copying or replication operations to further protect archived content. It addition we implemented more levels of verification for the integrity of data recorded on tape, thus guaranteeing that archive data can always be read even in the case of damage, wear and potential corruption of the magnetic media, by means of scheduled scanning operations to check integrity by reading specific data inserted when writing the data for each individual file.

Data integrity can be checked by using a hashing functionality, an algorithm that as a function of the writing of data of variable size, it generates a fixed-length “fingerprint” for the data added. This operation is performed for each file archived and the hash determined by the mathematical operation is then stored in the archive along with the file. The use of hash algorithms to check data is available for both the LTFS and TDO proprietary file systems. SHA1, SHA256 and SHA512 hash algorithms can be utilized.

Hashes are recalculated and compared during scheduled scanning to check media integrity. If a hash does not compare during verification, the associated file is marked as “bad” and another copy of the file (security copy) will be retrieved in real time in order to replace the corrupted one. In order to restore the correct number of copies, the second (or third) copy of the retrieved file will be made on a new medium.


LTO-8: Superior Performance to Hard Disks

Some vertical markets, such as video or high-performance computing, require extremely high sustained transfer rates due to very high capacity data sets, often in multiple petabytes. LTO-8 offers higher performance levels (350MB/sec Native) than the market’s fastest hard disk.

In order to properly compare disk and tape, it is important not to be misled by some published specifications (technical specifications are not always complete – for example write speed is often omitted).

There is an appreciable difference between the declared BUS data transfer rate and the effective one, i.e. at which the heads can actually write continuously to the surface of the disk’s internal platters before filling the on-board cache of just a few MBs (64-128 MB). A hard disk with 12 Gbit/second SAS interface and speed of 15,000 RPM offers a write data transfer rate of around 148 MB/second.

On the other hand, the data transfer rate for LTO drives does not vary between read and write, while for hard disks, as we have seen, write speed is almost halved (approx. −40/45%). Of course, in reality, multiple disk drives will be used in an array and multiple tape drives will be used in a tape library. Tape drives can be dedicated to specific read or write operations (unlike disk) allowing for excellent performance tuning for both information retrieval and online archiving operations.


LTO Roadmap


Tape Libraries: Best Cost for TB and TCO.

Today’s tape libraries provide the lowest cost form of storage in terms of both initial acquisition cost ($/TB) and running costs: electricity, cooling, operations & maintenance and technology refresh.

LET’S DISCUSS YOUR NEEDS

We can help you reduce cost, decrease vendor lock-in, and increase productivity of storage staff while ensuring accessibility and longevity of data.

Contact Us >