Scalog: Seamless Reconfiguration As Well As Full Guild Inwards A Scalable Shared Log

Corfu had the prissy persuasion to divorce ordering as well as replication. The ordering is done past times the Paxos box, i.e., the sequencer, as well as it assigns unique sequence numbers to the data. Then the replication is offloaded to the clients, which contact the storage servers alongside the sequence divulge to commit the data. This technique achieves scalability equally the divulge of clients increase.


A limitation of Corfu is that whatever alter inward the laid of storage servers makes Corfu unavailable until novel configuration has been committed to all storage servers as well as clients. Corfu requires all the clients as well as storage servers to have got the same mapping business office which maps sequence numbers to specific shard. This newspaper provides a uncomplicated (almost trivial) persuasion for solving this work as well as improving over Corfu to keep globally ordered shared logs alongside seamless reconfiguration of the log-shards.

Scalog  

Scalog turns the Corfu decoupling strategy on its caput alongside a judo move. By outset replicating the tape as well as thus assigning sequence divulge to the tape via a batch watermarking strategy, it solves the unavailability work Corfu faces during reconfiguration. (This also takes attention of the work Corfu faces  with a customer who took a sequencing divulge as well as crashed without replicating it, leaving a gap inward the log.)


In Scalog clients write records direct to storage servers, where they are (trivially) FIFO ordered without the mediation of a global sequencer. Records received past times a storage server are thus similar a shot replicated across the other storage servers inward the same shard via FIFO channels. Periodically, each storage server reports the lengths of the log segment to an ordering layer. To make a total social club out of these local/shard ordered log-segments, the ordering layer inward Scalog summarizes the *fully replicated prefix* of the primary log segment of each storage server inward a cut, which it thus shares alongside all storage servers.



The ordering is done at the Paxos box inward the ordering layer past times releasing version vector similar watermarks across sharded logs based on the fully-replicated log-segment progress heard from each log. The ordering layer interleaves non exclusively records simply also other reconfiguration events. As a result, all storage servers see the same update events inward the same order. The storage servers utilization these cuts to deterministically assign a unique global sequence divulge to each durable tape inward their log segments using the deterministic ordering within each.

Remark: Kafka also provides horizontal scalability of the logs via sharding. However, cheers to the version vector watermarking, Scalog tin sack impose a global social club on the logs which is real useful for many applications such equally those that require to do multi-object transactions across shards (as inward Scalog-Store application shown inward the paper). The global social club is also real useful for debugging problems across shards/subsystem, which many deployed systems run into.

As nosotros have got seen, this batch-based total social club imposition provides both seamless reconfiguration as well as scalability to Scalog. These ii figures explicate the Scalog architecture as well as performance real nicely. The aggregators serve equally soft-state buffer to batch communication inward front end of the Paxos box to alleviate the communication the box needs to endure.


In a storage shard, f+1 storage servers are sufficient to tolerate upto f crash failures. Due to its loosely decoupled coordination which tin sack handgrip whatever divulge of shards y'all throw at it, Scalog also takes an interesting approach to fault-tolerance. It uses ii servers inward a shard to tolerate the crash of ane server, as well as instead of trying to resuscitate the crashed server (which may have got a long time), it prescribes the customer to finalize this shard, as well as start a novel shard to proceed operation. After all, adding a novel shard is frictionless as well as no-effort inward Scalog.

Evaluation

Scalog is evaluated extensively as well as is shown to give practiced results. To evaluate Scalog at scale, the authors utilization a combination of existent experiments as well as emulation. They utilization a 10 Gbps infrastructure as well as SSDs, as well as consider 4KB records. With 17 shards, each alongside ii storage servers, each processing 15K writes/sec, they present that Scalog achieves a total throughput of 255K totally ordered writes/sec. Further, through emulation, they demonstrate that, at a latency of most 1.6 ms, Scalog tin sack handgrip most 3,500 shards, or most 52M writes/sec. This way that Scalog tin sack deliver throughput almost ii orders of magnitude higher than Corfu’s, alongside comparable latency. They also supply experiments to present how reconfigurations deport upon Scalog, as well as how good Scalog handles failures.

Discussion

Embrace small-coordination as well as punt things to the client

The master copy contributions inward Scalog are that:

1. it allows applications to customize information placement 
2. it supports reconfiguration alongside no loss inward availability
3. it recovers apace from failures

The outset ii features are generally client's responsibility. The affair most Scalog is that it provides small-coordination (as inward modest government), as well as it gets out of the way of the client, thus the customer tin sack customize its information placement as well as add together novel shards without leading to a slowdown or loss inward availability inward the system. The 3rd feature, recovering apace from failures, is also an artifact of the small-coordination provided past times Scalog.

How does Scalog compare alongside consensus protocols?

Scalog is non a consensus protocol. It assumes Paxos/consensus box to supply the strong-consistency/linearizability to the seem upwards of fault-tolerance. From the Paxos side, the closest persuasion to Scalog is SDPaxos. SDPaxos also separates ordering from replication, as well as replication is done outset as well as ordering is done over the already replicated requests. On the other hand, SDPaxos does non supply horizontal scaling equally inward Scalog. There must last a kind out SDPaxos overseeing each shard, as well as fifty-fifty thus nosotros volition require a similar watermarking push clit a fast ane on equally inward Scalog to impose a posteriori global ordering on the logs.

Scalog's ordering is a posteriori ordering. There is no immediate read performance supported inward Scalog. Reads are supported either through subscribe performance (which provides streaming batch reads) or via readRecord(l,s) operation, where l is the sequence divulge as well as s is the shard. In either case, it seems similar the reads are reads from the past times as well as non real-time reads. I wonder if at that spot is a loss of functionality because of this. For illustration does this elbow grease problems for log-based global RSM maintenance. The newspaper cites analytics applications for utilization of Scalog, as well as it is non real clear if nosotros could have got responsive ascendancy servers maintained using the RSM approach over Scalog shared logs.

How does Scalog compare alongside AWS Physalia?

AWS Physalia presented large-scale sharded configuration boxes which oversee sharded chain replication systems. I recall Physalia is a amend engineered organisation since it takes into draw organisation human relationship partitioning problems, as well as tin sack reconfigure the coordination cells to relocate closer to the storage servers to handgrip unopen to partitioning problems.

The batch-style asynchronous acknowledgements inward Scalog is bad for usability. They arrive difficult for the clients to create upwards one's hear when at that spot is a work as well as where at that spot is a problem. The clients tin sack non differentiate betwixt a work inward the shard as well as inward the ordering layer. Are the ii servers inward the shard partitioned from each other? Or is the entire shard partitioned from the ordering layer. Is the ordering layer functioning OK as well as having delays or is it downwardly or unreachable. These bound the actions the clients tin sack have got to response as well as adopt to problems.

Of degree Physalia does non supply an ordering across shards, simply possibly an ordering layer tin sack also last added similar to the watermarking ideas presented inward Scalog.

Share this post