Petashare: A Reliable, Efficient Together With Transparent Distributed Storage Administration System

This newspaper past times my colleague Tevfik Kosar (to seem soon) presents the pattern together with implementation of a reliable together with efficient distributed information storage system, PetaShare, which manages 450 Terabytes of disk storage together with spans seven campuses across the state of Louisiana.

There are 2 top dog components inward a distributed information management architecture: a information server which coordinates physical access (i.e. writing/reading information sets to/from disks) to the storage resources, together with a metadata server which provides the global cite infinite together with metadata of the files. management is a challenging work inward widely distributed large-scale storage systems, together with is the focus of this paper.

Petashare architectureThe back-end of PetaShare is based on iRODS. All organisation I/O calls made past times an application are mapped to the relevant iRODS I/O calls. iRODS stores all the organisation information, every bit good every bit user-defined rules inward centralized database, which is called iCAT. iCAT contains the information of the distributed storage resources, directories, files, accounts, metadata for files together with system/user rules. iCAT is the metadata that nosotros ask to manage/distribute inward PetaShare.

Multiple iRODS servers interact amongst the iCAT server to command the accesses to physical information inward the resources. Of course, the centralized iCAT server is a unmarried dot of failure, together with the entire organisation becomes unavailable when the iCAT server fails. As nosotros hash out next, PetaShare employs asynchronous replication of iCAT to resolve this problem.

Asynchronous multi-master metadata replication

PetaShare starting fourth dimension experimented amongst synchronous replication of the iCAT server. Not surprisingly, this led to high latency together with performance degradation on information transfers, because each transfer could last committed entirely afterwards iCAT servers consummate replication. To eliminate this problem, PetaShare adopted an asynchronous replication system.

The biggest work of asynchronous multi-master replication is that conflicts come about if 2 sites update their databases inside the same replication cycle. For this reason, the proposed multi-master replication method should abide by together with resolve conflicts. Petashare uses a conceptual conflict resolver that handles such conflicts. Common conflict types are: (i) uniqueness conflicts: come about if 2 or to a greater extent than sites attempt to insert the records amongst the same primary key; (ii) update conflicts: come about if 2 or to a greater extent than sites attempt to update the same tape inside the same replication cycle; (iii) delete conflicts: come about if ane site deletes a tape from database land around other site tries to update this record.

To preclude uniqueness conflicts, ID intervals are pre-assigned to dissimilar sites. (This could every bit good last achieved past times prefacing IDs amongst the site ids.) Update conflicts are handled using the latest write dominion if non resolved inside a day, simply at that topographic point is a one-day grace menstruum where negotiation (manual conflict handling) tin give notice last used. Delete conflicts are too handled like to update conflicts.

Evaluation

The newspaper provides real-deployment experiment results on centralized, synchronous, together with asynchronous replicated metadata servers. The no-replication column indicates using a centralized metadata server. Table 2 lets us to evaluate the performance of replication methods because the contribution of information transfer to the latency is minimized. For all information sets the asynchronous replication method outperforms the others, since both write together with database operations are done locally. Similar to Table1, the cardinal iCAT model gives ameliorate results than synchronous replication.

Share this post